Hacker News Change To Comment System Reminiscent of What Killed Digg
This is a previously unpublished report on the events that lead to Digg’s demise. I was reminded of it when I saw Paul Graham’s post to Hacker News about upcoming changes to the way comments work on Hacker News. This isn’t the first time Paul has talked about not liking the way conversations on the community work, but it is the first big change in a long while to the forum.
Personally I believe there exists the technology to automate moderation so that there wouldn’t be a need to create a high barrier to commenting, and could “train” the noobs and the trolls to comment in a better way, but that is a conversation for another forum. Follows is the case study of another popular forum that changed its dynamic and failed as a result.
A New Frontier or a Collapsing Startup?
Digg.com is a site that democratizes news by aggregating and displaying the most popular stories of the day. Users of the site also get social networking functions of adding friends and networking with stories dug by their friends. Digg.com is under speculation by the media now because of massive layoffs and reorganization of the startup. The company’s focus on stories in the media made them a media darling starting in 2005 – bloggers, journalists, and users spent more time driving digs to their stories and their friends’ stories than any other competing site. However, today Digg lost its position in the media due to a re-launch that caused numerous reports and speculation that the company is failing fast.
With 40 million monthly users, Digg has major online presence. Starting in 2004, Kevin Rose came up with the idea of “digging up” stories that resonate with the community. If the community collectively finds the story important and useful they can digg it up. Digg used to have an option where you can burry a story that you don’t like – this feature no longer exists because according to Digg it was an unsuccessful feature. We are going to take a look at Digg’s history to track the rise and fall of Digg.com.
The brainchild behind Digg is Kevin Rose. Kevin started the company along side Owen Byrne, Ron Gorodetsky and Jay Adelson with only a $6,000 angel investment from Kevin Rose. In 2004 Kevin named the company Digg with two “g’s” because Dig.com with only one “g” was taken and owned by Disney. Launching on December 5th, 2004 – Digg expanded quickly as a community based news aggregator. Rose came up with this idea because he was a reporter with his own show on TechTV called the Screen Savers – a show about popular culture and technology. He was always searching for popular news on sites like Reddit, StumbleUpon, and Fark, but wanted to use a site where the stories were voted upon making the most popular stories rise to the top. He also wanted to enable a search for the most active of his friends. He was able to leverage his fan base from his show in 2004 when he announced the launch of his site, Digg.com, which drove significant traffic and also helped by cross promotion.
In 2005, Digg released more features desired by the users including comments and profile pages. Then they introduced a backlog that would show the history of the stories you digg on your profile page. They launched Digg Spy and in the same year, updated it to spy 2.0. To further promote his startup, Kevin Rose launched a new show called Diggnation. Diggnation stars Rose and co-host Alex Albrecht. The same year Diggnation launched, Digg.com 2.0 launched with friend lists, page load management, and a new interface design. People were so happy with both launches – the user base grew tremendously. Digg raised $2.8 million dollars from investors.
In 2006 Digg 3.0 was launched. Digg 3.0 had categories that went beyond just technology based news stories. They opened up to sports, and science, world, business, gaming, videos and entertainment categories. Digg launched top stories with a #1 slot and the ability for users to “favorite” the top stories they enjoyed. In the same year, Digg launched podcasts, videos, and a sidebar.
Digg opened their API to the general public of developers who wanted to write cool apps and tools for the site, in 2007. They experienced a bad intrusive attack when hackers found a way to crack the HD-DVD encryption key on Digg and posted instruction manuals for other hackers to learn this hack on Digg.com. Although this drove traffic to the site – Digg had a major cleanup on their hands. They had to take down all of the hack information being posted in order to avoid any legal trouble. This led Kevin Rose to address the community. To many people’s surprise, instead of siding with taking down posts with the hack he sided with the hacker community posting when he said, “But now, after seeing hundreds of stories and reading thousands of comments, you’ve made it clear. You’d rather see Digg go down fighting than bow down to a bigger company. We hear you, and effective immediately we won’t delete stories or comments containing the code and will deal with whatever the consequences might be.”
2008 was the biggest year for Digg’s financial security when they raised $28.7 million from investors at Highland Capitol Partners. Then, Digg Bar was born – a toolbar that sits on the top of the page. The Digg Bar creates tiny URLs to the stories, comments, analytics, and so forth so that users don’t have to leave the page they are on. This bar was not well received. In the same year Digg integrated with Facebook connect allowing users to connect their accounts and participate in Digg without even having an account.
2009 is the year Digg launched Advertising on the site. Marketers were able to submit their ads and people were able to dig the ads they liked, but not able to bury the ads they didn’t like. In 2010 Digg launched an iPhone app and a little later an android app. Kevin Rose announced they would be removing the controversial Digg Bar upon the launch of Digg 4.0. The new user registration was taken down as they were testing Digg 4 alpha. Digg 4.0 launched this year.
Information Technology – Digg 4.0
In 2010 Digg made a decision to move away from MySQL because it was increasing difficulty of building a high performance, write intensive application on a data set that is growing quickly without an end sight. Digg wanted their engineers to implement changes to the database organically – rather than using MySQL to make generic changes.
Google and Amazon’s broad use of non-relational BigTable and Dynamo systems inspired Digg to make this switch. After lots of thinking and debating Digg decided to go with Cassandra.
Cassandra is a distributed database with a BigTable data model running on a Dynamo like infrastructure. Cassandra is a column oriented, allowing for the storage of structured data. It has a fully decentralized model – every node is identical and there is no single point of failure. It is also very fault tolerant – data is replicated to multiple nodes and across data centers. According to Digg Cassandra is also very elastic – read and write throughout increase linearly as new machines are added. Digg claims to have tested Cassandra on their live sites – the tests went well.
At the time of this transition Digg reimplementated most of their functionality using Cassandra as their primary data store – supplementing Cassandra-based indexing using full text, rational and graph indexing systems. Digg claims to have made massive performance improvement with using Cassandra. They have seen an increase in comparator speed, better compaction threading, reduced logging overhead, added row-level caching and implemented multi-get capability. In technical sense Digg tested and improved the operational capabilities of Cassandra. They upgraded its Rackaware capability, added slow query logging, improved the bulk import functionality and implemented Scribe support for improved logging. On top of that Digg claims to have done a ton of operational testing before re-launching Digg 4.0.
What Went Wrong?
As Digg 4.0 launched major problems surfaced. The main problem was – Digg placed value in Digg Ads over the value they invested in their users. The widget that allowed people to put their Digg activity directly on their computer dashboard was removed at launch of Digg 4.0. The site was not reachable and not stable during launch day and the weeks following. The new site was not well received by the users as they revolted in an out lash on blogs. Kevin Rose then responded in a blog post on his blog and promised to restore the upcoming pages and fix the broken algorithm.
Digg.com experience major glitches the first week of the site re-launch. Digg’s traffic dropped by 24% since the launch of Digg 4.0
Everyone was commenting on the poor site re-launch, including one of Digg’s major competitors – Reddit, who said, “This new version of digg reeks of VC meddling. It’s cobbling together features from more popular sites and departing from the core of digg, which was to “give the power back to the people.” Ian Eure, who previously worked at Digg as an engineer, said that old features that users loved can be implemented to Digg 4.0, but Digg didn’t add the features that the users wanted back. Instead they looked to external influences of the ad clicks with a higher value than the users. Digg’s product mix diminished this year as Digg started to crack down on power users. For example one power user – Supernova17, who was getting paid to digg stories for marketers without sharing revenue with the site, was banned. According to a blog post, the average cost for users to get the stories to the front sold to marketers for $500 a story. This caused a significant problem to Digg because they wanted marketers to go directly to the company instead of the power users.
Digg’s response to the users profits was to take away the power from the power users. Now Digg has partners like the New York Times or TechCrunch – these partners bring in content allowing users to select their favorite stories from their partners. Before it was about the users bringing in the content they like and now Digg is only allowing the users to use their selected partners, limiting the selection process. This change goes directly against Digg’s mission statement and their vision. Mission statement: “Digg is a place for people to discover and share content from anywhere on the web. From the biggest online destinations to the most obscure blog, Digg surfaces the best stuff as voted on by our users. You won’t find editors at Digg – we’re here to provide a place where people can collectively determine the value of content and we’re changing the way people consume information online.”
During this year Digg experience a big shift in management. For example, the fridge is fully stocked with a variety of beer. The employees are allowed to drink and work at the same time and smoking week is part of the culture. Kevin Rose wears marijuana leaves printed on his clothes and can often be spotted with a been in hand. Digg’s CEO, Jay Adelson, stepped down from the company this year due to constant fighting with Kevin Rose. Rose stepped up as temporary CEO until he hired Matt Williams who is now currently the CEO.
After trouble with a shifting management team and a buggy site re-launch the company is struggling to stay above water. Many people are speculating where Digg will go from here and what really went wrong with the users revolt against the site. That’s why we did some primary research with the former head of business development, Matt Van Horn, in a personal interview in San Francisco. This is what he said:
How did technological problems affect Digg?
Re-launch – We were very ambitious working with new technologies, there was a push pull of deciding what should be done. We were already late getting the new Digg out there – so we were trying to figure out what to do. It ended up taking a few weeks to get rid of the bugs and for us to switch form the old Digg to the new Digg. I think we moved very quickly to get it out there and I wouldn’t have changed anything. It took a few weeks to get it out there – but once it was out there, all the technologies worked really well together at the end.
Could you have done something differently while switching from the old system to the new one?
I was a supporter of: “get it out as soon as possible and we’ll figure it out later” it was rough on traffic that week, when we were having the most bugs. People were staying really late hours and on weekends to make sure the new site was running and working smoothly. If we spent too much time testing the new interface – we probably wouldn’t even be launched by now. I’m a big fan of failing quickly – if a feature doesn’t work iterate on it quickly.
Why did you decide to switch from the old technology to the new technology in the first place?
We were big supporters of new technology. We were working with Facebook and Twitter and other networks – they were all using Cassandra. We wanted to switch to keep up – and Cassandra is a lot more flexible.
Why isn’t Digg making money?
We’ve had A LOT of innovation with Digg’s ad platform – we have been sold out since last November.
What about your users? Traffic has dropped off Digg drastically – is the problem in the interface or is it the business model?
People loved Diggable adds – you also had a choice to bury an ad if you don’t like it – to never see it again. As far as traffic is concerned – we took some chances in moving Digg onto the new platform. We were very ambitious – getting people to follow the brands that they like. I’m not exactly sure what’s more affective. The team at Digg is very innovative and they will be able to figure out what’s working and what isn’t.
What about Digg’s old Power Users?
I know that many Power Users don’t like the new Digg. The publishers have too much control now – its not as rewarding to be a top digger as it used to be with the new Digg platform. I used to be a top digger before I came top work for Digg – Digg isn’t rewarding people as much now for curetting new content, because big blogs and news sources are submitting those stories to the new Digg platform now. Users aren’t as powerful on the new Digg platform.
The idea is that The New Digg wants publishers to curette content into the system – and the users can decide what the best content is from that mix. It’s still a work in progress and the content on the top Digg page is still highly rated content – the users submit that content. Digg is different – that’s what it comes down to.
On the old Digg platform most submissions came from the top 100 users – it isn’t so anymore. But I don’t think that a shift in power would drastically affect Digg’s traffic.
Where do you see Digg in the next 5 years?
I think that Digg was moving to be Very data driven and looking at lots of information and testing and figuring out what sticks and what doesn’t and then moving from there. There is a lot to be done – figuring out what interests people what users are interested and launch features accordingly. And if a feature sucks then Digg will get rid of it. The new Cassandra platform allows us to push code out very quickly. Before we didn’t push new code out hardly ever at all.
Does Digg have an IT department?
More than 50% of Digg are engineers.
Do you think that Digg’s business model will ever shift? Or will it stay consistent to the end?
Digg has always been about the congregation of content done by the people and the idea of wisdom of the crowds and that will stay core to the mission of Digg. We became very innovative with our ads and we’ll continue to innovate on that – so not only the users are happy but advertisers are happy also.
Why did you leave Digg?
I am a Digg loyalist and I was just ready to move on to the next thing. Digg was my first job out of college. I started out as a marketing assistant and I ended up running biz dev there and a new opportunity came up and I couldn’t say no to. It was time for me to move on to something smaller.
Digg recently fired 10% of their staff and now they are narrowing down the company further with the goal to reach profitability by lying off 37 percent of staff. The company has $15 M in revenue, but will slow down profitability with the cuts according to TechCrunch. Digg’s CEO Williams predicts they should be profitable by mid 2011.
As layoffs are happening key people are starting to jump ship. Chas Edwards, the chief revenue officer of Digg.com, is leaving the company. The well-known executive, who hailed from Federated Media to Digg in 2009, is headed to the photo start-up, Pixazza, which tags photos and images for advertisers so that related content can appear to match the images with similar products. Edwards’ new title at Pixazza will be Chief Revenue Officer and Publisher Development.
The Mountain View, CA based start-up is well funded from Google Ventures, August Capital, CMEA Ventures, Foundation Capital and Shasta Ventures. In addition, they have a small angle investment round from Ron Conway, Gideon Yu, and Maynard Webb. The departure is a good one, as Chas Edwards will continue on at Digg in a small way. He will remain an adviser to the company, which lately has been criticized by their users and the press in recent times. Since the new CEO, Matt Williams, came on board there was some management upheaval reported.
Williams had to apologize after stepping in the bugs in the site re-launch and promised to fix the issues. Matt Williams posted a blog post that explains how lying off staff will help to improve the site bugs and generate more momentum and business. Here is the blog post from the CEO, Matt Williams and how he announced the layoffs on the official Digg blog:
When I joined Digg six weeks ago, we set an immediate focus on improving the web site. We listened carefully to user feedback and started making changes to generate momentum in our business. As I mentioned in one of our first all-hands meetings, another top priority was to take a hard look at the entire business, across product, sales, and operations. Through the time I have spent with each of you, I’ve been impressed by the commitment and enthusiasm you’ve shown. I’ve also learned a great deal about what is working well at Digg, and what is broken. Many things are working well. The team is listening and acting quickly on the feedback from our passionate community. We’ve been able to deliver nimbly on the new platform, with over 100 bug and feature releases to the web site in the past two months. Our Diggable ads product has seen a notable increase in use by advertisers and clicks by users. Unfortunately, to reach our goals, we have to take some difficult steps. The fact is our business has a burn rate that is too high. We must significantly cut our expenses to achieve profitability in 2011. We’ve considered all of the possible options for reduction, from salaries to fixed costs. The result is that, in addition to lowering many of our operational costs, I’ve made the decision to downsize our staff from 67 to 42 people. It’s been an incredibly tough decision. I wish it weren’t necessary. However, I know it’s the right choice for Digg’s future success as a business. I’m personally committed to help find new opportunities for everyone affected by the transition. Digg’s Board members have also offered to help find placements within their portfolio companies. Let’s please use today to show our sincere appreciation for our friends and colleagues who will be moving on. Tomorrow, we’ll go forward with a new strategy for Digg.
It’s incredibly difficult to improve the product when the company is making such a major transition. Digg already lost their business development director, Matt Van Horn, to a new startup, Path, in August. Now they are loosing their CRO, Chas Edwards, and lying off 37% of staff. The management will need a successful transition to become profitable and change the company from the inside out. One would predict that the employees at Digg are going to be limited to innovate at this time because they will fear the uncertainty of their jobs.
At the end of October Digg was accused of faking Diggs to boost stories. 159 fake accounts were discovered – these accounts were boosting the popularity of stories from a specific group of domains. These domains included Digg’s publishing partners. Although Digg did acknowledge creating the bogus accounts, they claimed it was just a test. This was hard to believe because most of the stories submitted came from Digg’s founder and his girlfriend.
Digg.com is a private company who does not disclose their financial information, however, one would assume that they are struggling financially with the given layoffs and their focus toward an ad supported site. It would appear Digg burned through most of their $40 million VC and investor funding – putting all of their attention toward reaching profitability. Their planned action is to reach profitability by mid 2011. Given their push on ads it’s obvious that Digg isn’t focusing on their users. If they were to put all their attention to appeasing the users and keeping them loyal then Digg would have to devote a big chunk of their budget to that cause. We do not believe they have that budget or ability to give the users what they want because Digg is laying people off and focusing in on ad sales.
The clearest solution we see is to bring the focus back to the users from the advertisers, create an innovative environment through management, attract more diversity, and stick to the mission statement. The vision is clear with the company from day one so why is there a shift now away from the users to the ads? The site made too many major changes with the launch of Digg 4.0 and did not provide major testing before launching to the public. This left the users with a buggy site, which limited their ability to choose the stories they like to digg. Digg needs to focus more on providing the users with a site they gain value from.
Digg has to allow the power users to hold their place on the site rather than blocking them from the site and integrating more ads. The management should individually address employees to express their appreciation for their contributions to the company. The employees are probably afraid of loosing their jobs that creates limited to no innovation. The management needs to encourage innovation by being supportive of their employees so their employees are not afraid to step outside of the box. If Digg attracted a more diverse audience than their current user base – those new users may find a more rewarding experience on the site. We want to encourage more women to Digg because the current site is almost 100 percent male dominated. Finally, Digg has to stick with their mission statement. They need to keep the site about the users and not about the marketers. If Digg were to refocus back to their original vision and mission statement they will see more success and their traffic will likely restore back to what it was before the launch of Digg 4.0.
1. Marcus, Stephanie. “A Brief History of Digg.” Social Media News and Web Tips – Mashable – The Social Media Guide. N.p., 25 Aug. 2010. Web. 21 Nov. 2010. <http://mashable.com/2010/08/25/history-of-digg/>.
2. Cashmore, Pete. “The new Digg: Don’t believe the snipe – CNN.com.” CNN.com International – Breaking, World, Business, Sports, Entertainment and Video News. CNN, 26 Aug. 2010. Web. 18 Oct. 2010. <http://edition.cnn.com/2010/TECH/social.media/08/26/cashmore.new.digg/index.html>.
3. “Digg.com Site Info.” Alexa the Web Information Company. N.p., n.d. Web. 30 Sept. 2010. <http://www.alexa.com/siteinfo/digg.com#>.
4. INGRAM, MATHEW. “Digg Cuts Staff by 37%, Loses Senior Executive – NYTimes.com.” The New York Times – Breaking News, World News & Multimedia. New York Times, 25 Oct. 2010. Web. 15 Nov. 2010. <http://www.nytimes.com/external/gigaom/2010/10/25/25gigaom-digg-cuts-staff-by-37-loses-senior-executive-72501.html?partner=rss&emc=rss>.
5. McCarthy, Caroline. “Digg’s Matt Van Horn leaving for start-up Path | The Social – CNET News.” Technology News – CNET News. CNET, 26 Aug. 2010. Web. 18 Oct. 2010. <http://news.cnet.com/8301-13577_3-20014852-36.html>.
6. Parr, Ben. “What Digg Must Do to Survive.” Social Media News and Web Tips – Mashable – The Social Media Guide. Mashable, 10 Sept. 2010. Web. 17 Oct. 2010. <http://mashable.com/2010/09/15/what-digg-must-do-to-survive/>.
7. Quinn, John. “Saying Yes to NoSQL; Going Steady with Cassandra | Digg About.” About Digg | Digg About. Digg, 7 Mar. 2010. Web. 21 Nov. 2010. <http://about.digg.com/blog/saying-yes-nosql-going-steady-cassandra>.
8. Tsotsis, Alexia. “Kevin Rose Speaks Frankly About The New Digg Rollout (TCTV) .” TechCrunch . N.p., 30 Sept. 2010. Web. 21 Oct. 2010. <http://techcrunch.com/2010/09/30/kevin-rose-interview/?utm_source=feedburner&utm_medium=email&utm_campaign=Feed%3A+Techcrunch+%28TechCrunch%29>.
9. Yin, Sara . “Digg Accused of Faking Diggs to Boost Stories | News & Opinion | PCMag.com .” Technology Product Reviews, News, Prices & Downloads | PCMag.com | PC Magazine . PC Mag, 25 Oct. 2010. Web. 15 Nov. 2010. <http://www.pcmag.com/article2/0,2817,2371601,00.asp>.
10. “Digg: Five Years in 5 Minutes” Video. Nov. 2010. YouTube. 25 January