Impact of filters &...
Clear all

Impact of filters & rankers on standards

4 Posts
3 Users
Active Member
Topic starter

Decentralization of storage and prevention of censoring has to of course come first, but it seems by it self it leads to low adoption (because many people are ignorant & naive now of the dependence of their rights on free speech, even when they want those other rights strongly, and will not seek censorless platforms without added extra value to them, or until their rights are taken).

Currently, the customers are not the users but those who would wish to buy influence over the users using what you'd call ‘filters’. Doing this leaves them massive advantage over any (decentralized) system that morally objects, but it also ties their hands behind their backs. Their business being the sale of the user, they can only provide the user such a utility level of a filter before their market would collapse. This leads to stagnation of user facing utility while growing capacity to control through influence and means, I think, that the greater user control of the 'filter' is the key niche that would actually drive greater adoption if filled.

A 'standard' as it were would, as you say, facilitate this, but while any additional standards should be perhaps as simple as possible, I think it is very worth pondering: Do filters have anything that can be flushed out easily enough at a high level in their implementation to add usefully and simply to the standards, or provide a useful set of guidelines or suggestions to their implementation?

For instance, a git like storage mechanism has been mentioned for tracking providence, this would make blame and circular reference detection rather easy to better than ever expose to the user the means to filter that is currently withheld from them (this btw is what I mean when I say a personal 'AI' filter for each user, not terminator or some 'post singularity' device that actually understands what it's reading, though my hope for that one day is non zero, it's naive to consider here). Are there considerations that could be made in the decentralization of storage or it's "dns?" however that could be pre explored to guide compute efficient implementations of such 'non-naive' filters?

Filter by identity will help, but it is really only one possible metric in what can be an ansamble method. Additionally, it fails (alone) to serve the physically threatened, politically persecuted, or simply young (those who have yet to choose who they want to be) and yet with the anonymity those problems would demand, the filter must stand to orders of magnitude more the number of users, or denial of service and spam attacks will cripple the network.


Posted : 13/08/2021 6:08 pm
hampson reacted
Member Admin

I can't respond regarding the more technical aspects of implementation, but the desired outcome of the encyclosphere network is for third parties and individuals to create their own ranking and filtering systems for ingesting the larger network, similar to how RSS only pulls from requested sources. There will likely be prepared lists that a user could subscribe to if they trust the source of the recommendation, but of course they should be able to fully adjust these settings according to their own needs.

The content can be provided as meta information in the header which is invisible to normal HTML webpage browsers, but readable when the content is pulled into encyclosphere network for discoverability. These details can be used to initially categorize content within the network and help the filters associate with the right terms. Ratings may be different for each reader and will likely be stored by third parties as well since people will generally rate an article differently depending on perspective and/or experience. Beyond a simple rating I would submit a potential vector of 'completeness' and possibly use this weighting factor to suggest articles to improve on or further define what is contained so you can search for summarized sources or more complete sources depending on your needs.

Posted : 25/08/2021 10:13 pm
Tim Chambers reacted
Active Member
Topic starter

So, thank you for covering that here again (I'd drawn this too heavily for those who've been in discussions so far). I do understand that the founding design scope relies heavily on third party filtering and ranking, and is more it self centered on or starting with the standardization of encyclopedia data and it's decentralized retention (Think that captures it?).

Big problems! (one's I'm concerned at least are a bit more than RSS, but we'll see where discussion goes; being the point, no?)

I don't want to jump too far ahead of that (actually, bias up front: I do. 🤣 I love your completeness idea and would personally like, as a user, to search by a visual selection in topic space and perpendicularly, visually, on the social graph of decent of a topic group, rather than just by key word search).

However, be it standards on such pre-filter tasks, examples of their implementation, or a full fledged implementation (however far it's taken), all are essentially a product. I don't say that to imply capitalizing on it, but rather to strongly stress: your 'user', if that is the scope, is the creator(s) of the filters, ranking systems, and viewers more than the users, because (and I should have captured this problem above, even if 'obvious'): Spam! And... Discoverability given the non cross domain nature of would be 'rankers'.

Simply, without moderation, a filter, or rankers of some form, it would quickly become nearly hopeless or useless to a user. (People will literally flood it  to break it because they can. In addition to just writing more than I. And more.) Again, I think that tool should keyly be in the hands of the user, but humans don't tend to fill that task well alone and where they do, they fail to generalize well. For instance, I'm a Roboticist with heavy previous focus on software; I can find many sources I would 'trust' for software, electrical, and mechanical engineering, but biologists (for inspiration), entrepreneurs, or sociologists? Without *being* an expert first, how do you find who or what you can trust?

Those each are a very (overly?) large esoteric problem, but those who have tried to 'fix' it 'for us!' have had to grapple with it, and have made it their business to do so (making the user for them into the product). Anything to come after must then, I think, solve that problem "better" (as measured by the user!) or it would be measured near useless to most users. That leaves some form of open filter & ranking developers the direct customer of KSF, as I understand it.

You could say even it's people only you're interested in doing those tasks (I think 'more' is needed), but in any case, paying close attention to their (would be) needs, in terms of exactly what in that spec should be, particularly in order to scale *their* performance, is probably rather prudent. If even pre considering the technical implementation is too much, seeking a maximum utility input from those who would, would I think really help ensure greater future adoption and usefulness.

Would really like to see actor diagrams a part of early seminars for this reason!

Posted : 26/08/2021 6:50 am
New Member

The post (I haven't read your comment carefully enough) presents an interesting view on the topic.

It views the distributed universe of the data as a whole that is consumed via filters.

I see the data differently  - I see it as disparate data points that are consumed via aggregators.

I believe this is an important distinction because it will bear on what the specification process is supposed to specify.

For instance - I note that you see as part of this spec the storage layer: "it's decentralized retention" which fits in well with your conception of the data comprising a universal (interconnected) whole that is consumed via user controlled filters.

According to my conception of disparate data points being consumed via aggregators, however, distributed retention need not be be specified because it's automatic and can be taken for granted.

That's a bit theoretical but on a practical level I also don't think it's feasible for this project to tackle a specification for decentralized retention.

Just as I believe users should be able to layer their own high level semantic specifications on top of the fundamental basic topical knowledge data format specification, I also think users should be able layer our data specification on top of any storage layer protocol or standard they choose. 

Posted : 26/08/2021 4:26 pm
hampson and Tim Chambers reacted