kluster’s decision engine — some explanation
by hitch - June 13th, 2008
hello all,
first off, everyone at kluster reads your feedback and us developers/statisticians/tech-heads are incorporating it as fast as possible.
second, thanks for being a part of these lab experiments and this community. we are very young (beta) and are refining our sites, our ideas and our decision engines.
/insert segue
i’ve pulled myself from the dredges of algorithm code and i’d like to share some insights with you all and hopefully clear up some of your concerns.
the algorithm strives to identify what you (the community) have decided is the best outcome. it is a decision support system, support being the operative word. we are constantly looking for holes and especially at this early stage, we expect to find them. once found, we assess, investigate, react and push out a newer version. the most important thing you should know is that the algorithm does not simply look for the idea with the most total investment. there are lots of variables, like WHO did each action, WHEN the action happened, and WHAT the action was.
WHEN is particularly important. If the selection process was as simple as overall popularity it would be very easy for users to “snipe” winning ideas, meaning that it would be easy to see which idea was going to be chosen and jump in with a big investment right at the end to ensure yourself a payout. We’re always concerned with ensuring your participation (investment) add value to the decision making process or the community as a whole, so we’ve attempted to correct for that behavior.
To assess the when and what, we run the data that are output through a series of filters. The filters are statistical tests that look at the distribution of investments in terms of amount, time and uniqueness, and then asses a likelihood that these (and other) variables are reflective of true community support. When these tests are done, the algorithm collates the data again and then runs a few final tests that rank ideas based on a number of variables that have inherent importance values.
One important thing to note (@josh) is that the timeline is relative to the ‘life’ of the idea and the stat tests judge based on that. therefore, it is entirely possible that an idea that is late in the game could win, provided it had support and valid investments based on time and amount and unique users count (not all the variables, but some to give you an idea).
We’re working with it guys, and always trying to improve the way it makes decisions. just today we realized (thanks in part to your feedback) that we had too much flexibility in the part of the algorithm that asses uniqueness. this has since been fixed. this comment thread and the feedback (and bringing to our attention any suspicious activity) is a perfect forum for improvement.
Thanks again and i’d be more than happy to field any questions.
hitch
comments
our sites
recent comments
- jeu casinos: COOL !!!
- John1092: Great site. Good info
- Igal: Card loading problem finally fixed, at least over here.
- Billy Starr: I am starting to think Kluster doesn't love us anymore. I have e-mailed payments 3 times and Team once and still awaiting a response. Maybe
- Eric: Payments are still down on my end too. Can't load the card for the past 2 weeks.
the silence is deafening, hitch. i guess that means you and your algorithm are perfect!
*grin*
Looks like putting a blinder on people makes the rewards the come back much sweeter. How does one go about keeping people on the train? Even HSX has a “set me back to 1000 points” button for people who hit rock bottom. Can something like this be done without it being exploited? (not that I’m at zero, this is just more interesting for everybody if a lot of people are voting and the community doesn’t bankrupt people perminantly)
Can anybody think of any good method of thinning out the names over time? So older names that do have (many) votes maybe expire early? Maybe this happens when the list is longer then (50) (100) (200) (300) (400)?
I always think it’s a bust when names like, innovention (no offense to its creator) win a place in the winners circle. Disney has been using innovention for ever and it’s fully unoriginal. Did you guys ever think of taking away the computer and fancy math and having the people paying the money choose the winners?
I’ve got Honeyshed on board to have the NameThis community choose a tagline for them, we’ll be posting on Monday. Loving the service, blogged about it today, but I think some of the winning names are boring.
-zack
Many people keep recommending that the entities utilizing the service should be the ones that pick the winners.
This website would serve no purpose if that were the case.
Namethis.com provides a service to its clients for $99. This service is a set of three names that have been validated by the consumer market. If the client chooses their own names they should not be paying ANY money, because none of the names will have been validated by anyone but themselves. They could simply go to any number of very active online forums and submit their request and undoubtedly receive tens if not hundreds of suggestions for free. Those submitting names for free may be less inclined to duplicate, steal, go off the wall, etc…when not trying to get a piece of the money pie.
The goal of this site is to provide a revolutionary service, not to provide ways in which its members can be chosen for their submissions to make money- that is just a fun incentive on the side. An algorithm is therefore necessary for Kluster to do what it is meant to. There is no question that Hitch will continuously be at work on the site, streamlining the selection process. The real issue exists with the users. We have to make a complete change in the way that we view this site and the entire process. Whether is be through a multi-phase system, or through a hierarchical suggestion format we must provide the client with this: three consumer validated names in 48 hours.
Algorithm needs to stay, though we all know that more changes need to be made.
drew… hells yeah man. you got it.
@drew +10,000 watts … just kidding, but spot on comment. thanks.
Aubrey Said “Even HSX has a “set me back to 1000 points” button for people who hit rock bottom. Can something like this be done without it being exploited?”
I agree, something needs to be done to keep users from leaving if they hit zero. I also noticed a Knewsroom post about the possibility of purchasing watts. I don’t agree with that wholly, but what about purchasing watts only out of your Kluster earnings? While I love the idea getting paid for contributions, and I probably would not pay out of pocket for watts, I would however exchange some of my Kluster money for watts. Maybe only have the option available for people with under 5,000 watts or so, and cap it at 5,000 max total after purchase to keep people from massing influence with money. I’ve been sitting at zero for a little while, and I am still contribution news links and namethis ideas, but the fun of investing is gone. Seeing as how this is directly related to customer retention, I would make finding a solution top priority. It’s much harder to find a new customer then to keep an existing one.
As far as the namethis algorithm, I think it’s fine as stated. It will shine when namethis becomes more popular and is receiving enough investors and original ideas to keep the gamers in check.
@drew + adam. very succinctly put. thank you guys
Thanks for the explanation.
While I think it is a great idea, I think the implementation may need a bit of tweaking.
For example, I usually go on at night after work and dinner, which is between 10pm and 12am. Now, one should automatically assume that no matter how many watts I invest, my portion/share should be significantly lower than those who invested early in the morning. But as it stands right now, I only get my investment back. As soon as I noticed this, I completely lost interest in investing and quit visiting knewsroom until today.
It’s impossible for me to get onto knewsroom earlier in the day (this is early for me, I’m still at work).
So why not adjust it so that instead of receiving NO reward, those who visit later and *snipe* receive a smaller reward.
Aside from that, I’ve heard others complain that they invested in stories that were eventually published, yet they lost their watts completely, this has also made me discouraged from investing because I believe it is probably a bug and won’t invest until I feel safe enough to invest again.
Hopefully these comments where helpful, thanks for job you guys do!
- David
As far as the namethis algorithm, I think it’s fine as stated. It will shine when namethis becomes more popular and is receiving enough investors and original ideas to keep the gamers in check.
-this makes sense.
Hey, Hitch, do you think we could do that cool yellow thing on blog posts that are new since we’ve been on? Or a little bookmark or something? I keep forgetting which threads I’ve read and how far down I’ve gone. It’s made getting through 600+ names on the Snack project much easier–thanks!
@ David - I believe the reason you are not getting a lot of watts back on investments is because most people are not losing watts, not because of the time of day. From my understanding, the winning watts are takes from the investments in stories that did not get published. Until the amount of stories and losing investments increases, the winners will not have a large pool to share from. I have large investments that only fetch 1 or 2 extra watts on top of the return. (Someone from Kuster correct me if I am wrong)
Both the Knewsroom and Namethis will have better returns as more people get involved in the sites. Remember this is beta, pretty much meaning in experimental stage, so just try to have fun while the kinks are worked out.
Thanks for the post hitch
@Adam : I agree that there is something that allows people to get some watts back to start investing again. I have some friends who I invited and lost all their watts almost immediately and now they are basically stuck with so little watts and since there aren’t enough “losing watts”, they can’t win back their lost watts and aren’t as active anymore as they were before.
If I can suggest something for namethis..can we have like a random suggested name generator ? for example there is a button we can click that also us to randomly see and invest on already suggested names. Scrolling is a bit tiring and I think people will stop halfway especially if there are 600+ names. instead of scrolling down people can just click a button and randomly see suggested names and once they find something interesting they can invest. It allows some people to focus more.
Also can there be more opportunities for us to earn watts ? recently my investments are only getting between +1 to + 20 watt rewards which is almost none already. If that can’t be done, at least maybe put up even artificially a pot of watts that can be won everyday in knewsroom.
@adam: yes, when more stories lose, more watts will be available to the winners.
I invested in the topic ” McCains Report $250,000 in Credit Card Debt” which made it to the front page, yet I didn’t even get my watts back returned let alone rewarded? What’s up?
@angie. is this still the case? let me know and we’ll further investigate
Angie and hitch - I had the same issue and was guessing it was because the story was under the 500 word minimum. If that’s the case, it would be good to have stories that don’t meet the guidelines yanked before they can be invested in. Maybe something that automatically counts the words in the submission box and doesn’t allow submission until 500 words is reached?
Does anyone else’s message inbox get “empty” messages? I suspect its for when my sparks are amped, but can anyone confirm this?
@hitch, yeah this is still the case.
Wondering the same thing:
Kristen Shamis said: Just received the card- thanks! How do I transfer my kluster money onto the card?…
@hitch, me too on the McCain story.
Do we really not get watts for a story that made the front page if it’s not 500 words?? I never thought to count the words… can’t the story be pulled before it makes front page? Or flagged so people don’t invest in it? I think we should get the watts if it makes it to front page….
this is becoming not as fun anymore
I feel like all I’m doing is loosing watts. I was loving knewsroom and trying to get all my friends to join. I really want this to work!
@ angie and stacy. you (and others) should be good now. took a bit, but we fixed the problem. still tell your friends and have fun??
I’m trying (still) to understand how this works. Biggest question (though I have many) is how one gets cash for proposing or investing in a story.
Second biggest question: There doesn’t seem to be any practical difference between investing in a story and investing in a topic. How does the algorithm weigh the two?
Third biggest question: How could I have 0% influence on today’s knewsroom when I proposed several stories that were “published” y and for which I earned watts? I realize the algorithm isn’t reality-based reality, but you can see how that would be confusing.
On the plus side, I’ve finally figured out how to win watts (but, alas, not enough to make it worth the risk)!
Thanks.
thanks
I had the same problem with the McCain Story and the problem was that there was a duplicate story. …Never got my 2040 watts back. SUCKY!
Yo @Hitch, I work for Honeyshed.com, and we just finished running a namethis campaign for our tagline. It was very fun to see all the names come in, but I’m not sure there was much satisfaction in the names that the algorithm picked. One thing is that we specifically asked for no names with buzz/bees/bears and such, and that’s exactly what we got.
I understand that some companies are coming to you for a solution that gives them a direct & final answer, but I have a feeling that a lot of clients might end up choosing a different name from the list than the ones that your algorithm decided were the “winners”. This is okay, because the company has a long list of options to choose from, but it seems unfair to the person who REALLY won.
Something I might suggest is to give the client the option during the sign up process of whether they want to choose the winning names, or let your algorithm do the work. I think you would find that people will still use the service because they aren’t as interested in being handed a WINNING NAME but more interested in direct & easy access to your pool of creative talent.
@Zack…Thanks for the client feedback! Most helpful! I was wondering about those bees.
Ah, but I can hear mattit and Ben and Hitch saying, gee, we want “market tested” results. Which is why we need a two-step process! Step One: We get the pitch and throw our brilliance at the problem. Step Two: We do a preliminary vote (so the client can see what direction the market is tending. Step Three: The client picks the top 25 of ALL the submitted names. Step Three: We vote (but you can’t vote for your own; blind submissions would make it harder for friends to support each other and skew the results). Step Four: Now we slap each other on the back and say “good job”!
Whatya think?
Oops–make that fourth step Step Four.
That is spooky. I was thinking overnight about a similar two-step solution. Twenty-four hours to pitch, twenty-four hours to vote. Submissions must be anonymous, you can’t invest in your own name and the site needs a moderator to sort out problems such as domain names, name-stealing and gaming.
This occurred to me mainly because of the environmental charity pitch: no disrespect to the good work done by the winners, but Green Cross, for instance, is already a long-established environmental organisation, GEC stands for General Electric Company and Econauts is already taken as a domain name. No organisation/brand wants to sound like another, already established, one. I’m sure half of my pitches fall into this category, hence the need for a moderator. We shouldn’t have to do the research ourselves because this is a brainstorming activity as well as a crowdsourcing experiment, and we need to keep the ideas flowing.
…and furthermore, I would say that although kluster only charges $99 for the service, serving up unusable names gives it a bad reputation.
@ zack. thanks for the feedback. i understand your concern, but remember that the algo picks winners based on people’s participation in an idea. meaning, if people submitted and subsequently invested in ideas that were not congruent to you (the clients) description and needs, the algo will pick the winners from that pool. to reiterate: it is a decision support system. it analyzes the user participation and statistically determines which submissions showed most community support. it is unfortunate that people clearly didn’t read your criteria well enough.
we are taking into account your suggestion for a client option aspect and will be discussing that further.
thanks again for the feedback and suggestions,
hitch
do we not get our watts back for the mccain story? I thought that’s what you meant by fixed… just wondering.
@hitch
Perhaps you could implement a system that takes into account some influence of the client (say 25 client/75 community). I also think Stacy has some really great suggestions - multi-phase voting I think is a must.
Perhaps there could be two reward sets - one for top two community chosen names and one for client chosen name (change the reward structure 35 client-choice, 35 top community choice, 10 second place community) If client chooses top names, you get an awesome reward.
Also, I have to put some doubt on the accuracy of the algorithm in analyzing community support. While I believe most of my wins were a result of community support (based on my relative percentage of the final 20%), My last two wins show that my share was 99%. I am not really sure how this is possible, as I have never invested more than 1000 watts in any of my own suggestions (the most recent of my wins, Spectate and Club à la Mode received 650 and 947 watts, respectively). So that means that community investment in Spectate must have been something like 25 watts and around 50 watts in Club a la Mode. Yet Club a la Mode came in first and Spectate came in second in their respective threads.
What I don’t understand is that the second place name HauteExclusive, has a much larger investment from its submitter (2355 watts, more than double my own investment) and I can find at least one more example with an investment of 1000 watts from its submitter.
I certainly care that winning names are reflective and I will return to making small investments on my own suggestions, but there seem to be some real flaws that need to be ironed out.
annoyabot.
jtsfds122 please leave and stop posting
Is there any way to block jtsfds from posting?
Yes please can something be done ??
cool site