Timeline for Are high-reputation users answering fewer questions?
Current License: CC BY-SA 4.0
68 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Jan 18, 2021 at 12:15 | history | edited | CommunityBot |
replaced http://data.stackexchange.com/ with https://data.stackexchange.com/
|
|
| Mar 20, 2017 at 9:34 | history | edited | CommunityBot |
replaced http://meta.stackoverflow.com/ with https://meta.stackoverflow.com/
|
|
| Apr 1, 2015 at 17:56 | comment | added | David Robinson | @Wooble At long last I can now add those freehand circles: stackoverflow.com/q/29397338/712603 | |
| Sep 17, 2014 at 4:58 | comment | added | Shog9 StaffMod | I missed this conversation, David & @Rachel - sorry for being unclear. I was talking about total answers posted, which has decreased somewhat but not nearly to the degree suggested here (at least... at the time I posted that comment; things have changed a bit since). Answers per user is another story entirely - let me know if you're interested in the raw data for that. | |
| Sep 17, 2014 at 1:45 | history | edited | David Robinson | CC BY-SA 3.0 |
deleted 128 characters in body
|
| Aug 20, 2014 at 9:52 | comment | added | jay_t55 | It's a shame that higher-rep users are beginning to answer questions less frequently, a part of me thinks it may partially have something to do with dumbos like me who ask too many dumb questions; for that I am sorry. Please come back. | |
| Aug 9, 2014 at 7:28 | history | edited | Sam | CC BY-SA 3.0 |
deleted 145 characters in body
|
| Jul 3, 2014 at 15:43 | comment | added | Rachel | @DavidRobinson Ah I guess that's true. I was looking at this question more as "Are we getting fewer answers from high-rep users", as opposed to "Are we getting fewer answers from our existing group of high-rep users". I suppose if viewed in the form of the second question, your answer would be correct, however if viewed in the form of the first question, I think it would be incorrect :) | |
| Jul 3, 2014 at 15:38 | comment | added | David Robinson | @Rachel: It depends what question you are asking. I was taking a class of users (users who currently have >20K reputation) and asking whether their answering activity had decreased in the last two years. Actually I think Shog's analysis is much more misleading, since he's looking at the # of answers contributed by a changing population. If the number of answers posted by 20K users is static even as the population is increasing, that does mean their individual contributions are decreasing. (Unless he is saying the number of answers per user is static, but that's not clear from his comment) | |
| Jul 2, 2014 at 14:26 | comment | added | Rachel | This answer is wrong as Shog explains here as a comment to my answer : "Out of curiosity, I ran the numbers for folks with 20K+ at the time each answer was created. As you predicted, Rachel, the results were rather different - indeed, the number of answers posted by 20K users was fairly static over the years, even as the number of 20K users increased." Your graphs take into account Reputation as of now, which skews the numbers by quite a bit. | |
| May 22, 2014 at 2:32 | comment | added | Erwin Brandstetter | Great answer overall. Still -1 from me for sensational, misleading graphics that are no 0-based just to convey a more dramatic (false) impression. I hate that. | |
| May 21, 2014 at 19:44 | comment | added | Anthony Pegram | I have one or three reputation points. I've essentially stopped even coming to the site. Haven't answered a question in almost a year, haven't commented since last August. I'm just beyond fed up with duplicated, low quality, poorly researched content and would rather rage at the askers because there is not much else to do about it (or, rather, not enough people putting out the fires). I eventually just decided it would be simpler to just leave. But I wouldn't count myself as an expert, far from it. | |
| May 12, 2014 at 17:09 | comment | added | Henrik Erlandsson | @MartijnPieters is correct, and it's the same as in f.ex. emerging market. Get in early and the money keeps ticking in, from f.ex. already being known and already being big. The one coming in late has to work that much harder. Once you're established, there's no need for that original busy earnin' - and I don't see that as a problem. Likely the jobless knowitalls have finally landed their dream job that actually requires knowing it all, and we should all be understanding of their suffering and not accuse them of not contributing to the community as they used to. | |
| May 12, 2014 at 9:45 | comment | added | VonC | Yay! Still going strong after 5+ years :) | |
| May 8, 2014 at 14:50 | comment | added | Patrick M | I love this answer for being the Entertainment Tonight of StackOverflow. I may not know what Paris Hilton is eating for dinner tonight, but I sure as hell know Jon Skeet's posting habbits for the last two years. Thanks, David! | |
| May 8, 2014 at 14:48 | comment | added | Aaron Bertrand Staff | @joel3000 if you have a good question but it's already been asked (and answered), why do you need to answer it again? What would be the purpose in forking the sites and having that same good question answered differently in different places? If a more modern answer is better than the existing answers, add another answer. | |
| May 8, 2014 at 3:40 | comment | added | DVK | I'm not a super power user (60k only). My reason was actually mostly (though not exclusively) Stack Exchange's fault: I got overly distracted by non-SO new SE sites. Another reason was "Summer of Love". I hated the idea that SE was explicitly coddling help vampires while wagging their fingers at people who try to resist them - that demotivated me from moderating OR answering to an extent. | |
| May 6, 2014 at 18:56 | comment | added | joel3000 | As time goes on it gets harder to ask good questions because most of the good ones have already been asked. Old timers had the advantage of getting here first and answering the good questions back then, which pay interest now. One answer would be to start SO over again, in terms of questions. Keep the prior round, but let new questions and answers come up that will have more modern content. I don't know if that's a good idea though, it's just an idea. | |
| May 5, 2014 at 13:12 | comment | added | durron597 | How hard would it be to come up with a similar graph to the first one that includes 5k - 19999 rep users? I know that I've become jaded long before I hit 7200; in fact I am certain that I got jaded shortly after I earned the right to close vote. I'm wondering how similar it would be for other users. | |
| May 5, 2014 at 13:11 | comment | added | DevSolar | I found a gripe with the statistics presented: The observed changes could readily be explained by the increasing popularity of SO. Consider the hypothesis, "by the time you get to read a given question, the 'correct' answer has already been given". You only look at >20k users. Perhaps <20k users have been answering fewer questions as well, because of an increasing users vs. questions ratio. | |
| May 4, 2014 at 1:17 | comment | added | Andon M. Coleman | Is it really a good idea to sample individuals on the basis of their current reputation and infer that far backwards historically? A more useful query, were such a thing possible, would be to look at the number of answers people gave when they first attained 20K reputation versus now. For the better answerers, reputation grows solidly over time for answers they wrote a long time ago. Many users can hit the 200 rep a day cap just from random up-votes on answers written years ago. Because only accepted answers can break that cap, said users have to write fewer, higher quality answers. | |
| May 3, 2014 at 3:04 | comment | added | Rachel | @DavidRobinson I ended up posting an answer after all... just started typing and before I knew it I had something long written so I figured I might as well post it. So there you go, another opinion :) | |
| May 3, 2014 at 2:45 | comment | added | Rachel | @DavidRobinson I considered it, but there's already 25 answers posted and I didn't want to read them all to see if it was a repeat opinion of someone else :) | |
| May 3, 2014 at 2:45 | comment | added | David Robinson | @Rachel I think you should provide that as an answer, as it offers a contrasting perspective (and I'd be interested in seeing your complaint on elitism in more detail) | |
| May 3, 2014 at 2:42 | comment | added | Rachel | I'm a > 20k user, and personally I just don't have as much time as I did a year or two ago to answer questions. And if you checked with me 3-4 years back, I was still a user but very inactive as I didn't have time then to participate either. If this changes in the future, chances are you'll see me answering more questions again. I like helping people, and enjoy teaching newbies. If anything, it's the rudeness and elitist attitude of some users that will drive me away, not question quality (as it already happened on one site. :) | |
| May 3, 2014 at 2:39 | comment | added | Rachel | People change. Lives change. Its hard to stay part of any online community for years with the same level of involvement. In 2 years time, we'll have a new batch of 20k users, and their answers will increase the numbers displayed in the later half of your graphs simply because their questions answered now will then be considered "> 20k". Most users with 20k rep have been here a while, so its natural that older data has higher numbers. | |
| May 3, 2014 at 2:33 | comment | added | David Robinson | @Rachel: I certainly agree it would be good to look at the activity of users after they reached 20k (data that is unfortunately beyond my skills to extract from the Data Explorer). However, I disagree that "of course you're going to see a downward trend." The set of users that have 20k rep now is a fixed, defined class of about 2300 users. This is evidence that the contributions of those 2300 users has declined (specifically a subset of 47% of them). In what way is that tautological? | |
| May 3, 2014 at 2:17 | comment | added | Rachel | I think your graphs are misleading because it pulls data from users who have 20k rep today. You need to find a way to get users who had 20k rep at the time of answering, or of course you're going to see a downward trend like this. There are more 20k rep users now than 2 years ago, so older answers posted when the user was < 20k are going to show up in your graph, even though they shouldn't. | |
| May 2, 2014 at 15:10 | comment | added | joran | @tcaswell That would be a nice way to plot it, but David's way is fine too for the purposes of this post. I don't share your mistrust of the trend lines. Only 9 were actually claimed to be significant, and just eyeballing the graphs I count 7 that seem perfectly plausible to me. | |
| May 2, 2014 at 14:43 | comment | added | tacaswell | @joran If you want to suppress zero to show a small change on top of a big signal than you should plot (val - offset) and label the axis as such and make it really obvious. I also don't really believe many of those trend lines. The data is super spiky and they look way too biased by the end points. | |
| May 2, 2014 at 14:19 | comment | added | joran | @tcaswell The start at zero rule isn't as universal as people think. For the paneled plot, it is simply a judgement call on the intent of the graph: equal scales is better for making comparisons between people, separate scales are better for quickly seeing detail for each individual (which was the intent here I think). There's nothing wrong with David's choices here. | |
| May 2, 2014 at 13:41 | comment | added | Dave Newton | Ooo, I also want to see how I fit that. I know I've had several acute incidents which have caused me to slow down, and now essentially halt, any question-answering activity. | |
| May 2, 2014 at 12:32 | comment | added | user568109 | @RobertKoritnik Some of the points in question already answer that. But he did provide statistical correctness to the hypothesis. It is difficult to answer without the users themselves weighing in. They should be surveyed about what the want, what happened. To me it is like economy, pumping rep(money) to system needs to be controlled (in a manner to speak). Two users may have same rep but never the same experience/grasp/knowledge ... . In this way rep looses its meaning. They no longer care about rep(gain/reward/income), which is the reason to run an economy aka the site. | |
| May 2, 2014 at 12:21 | comment | added | tacaswell | It really bothers me that you suppress 0 in almost all of those graphs and plot them all on different scales. | |
| May 2, 2014 at 5:23 | comment | added | Robert Koritnik | @DavidRobinson: you are aware that you didn't answer the question though? You just (albeit plausibly) proved the question to be true. | |
| May 1, 2014 at 8:52 | comment | added | user541686 | @BoltClock: Yeah I feel similarly. | |
| May 1, 2014 at 5:12 | comment | added | A5C1D2H2I1M1N2O1R2T1 |
@SimonWhitehead, David links to his code used. If you know how to use R (particularly with RStudio), you should be able to download edit this: dgrtwo.github.com/pages/tumblr/are_users_quitting.Rmd. Towards the end of the file, you'll find plot.users("712603"). Change the user ID to yours and you're good to go.
|
|
| May 1, 2014 at 0:42 | comment | added | Simon Whitehead | "Ask and ye shall receive"... can I see my graph? How do we access these? @DavidRobinson | |
| Apr 30, 2014 at 10:12 | comment | added | user2140173 | I am sorry for possibly missing the whole point of analysing answering. IMO the problem is with the questions - more experienced users are not so keen on answering every single question they read. SO got stricter on closing poor questions and there is no point in answering 1. because it's against the SO rules 2. question gets deleted so any rep earned is subtracted...Look up how much professionals charge for consulting | |
| Apr 29, 2014 at 23:36 | comment | added | David Robinson | @Edward: One issue is that the number of upvotes is confounded with other factors- for instance, a common complaint is that SO is too ready to upvote bad questions. This is part of the reason I focused on high-rep user activity as a separate question, but some of the answers here, including this graph, do address this issue. | |
| Apr 29, 2014 at 23:33 | comment | added | user289086 | @Edward there is the quality filter. While only SE knows the full mechanics of that, getting a 'reanalyze all the questions asked for one day each week since the start' could give some insight into that. | |
| Apr 29, 2014 at 20:43 | comment | added | Edward | I wonder if there's a way to do a similar kind of numerical analysis on the quality of the questions. Perhaps based on upvotes per unit of time? That might allow one to test the hypothesis that questions are getting worse. | |
| Apr 29, 2014 at 20:35 | history | edited | David Robinson | CC BY-SA 3.0 |
Update link
|
| Apr 29, 2014 at 20:15 | comment | added | jscs | Thanks for clarifying, and for the analysis in the first place! | |
| Apr 29, 2014 at 20:14 | history | edited | David Robinson | CC BY-SA 3.0 |
deleted 2 characters in body
|
| Apr 29, 2014 at 20:13 | comment | added | jscs | I see, @DavidRobinson; I had understood this as "had 20k at time of posting", as Joran was talking about in the previous comment. (Do you mean "answers" in your comment and the sentence directly above the first graph, rather than "questions"?) | |
| Apr 29, 2014 at 20:08 | comment | added | David Robinson | @JoshCaswell: Not quite, because this analysis uses the set of users who are currently 20K, including the questions he asked before he hit 20K. So if a user asked many questions then hit 20K in January 2013 and quit, he would contribute to a strong downward trend (he'd add many questions to 2012 and none to 2013). | |
| Apr 29, 2014 at 20:06 | comment | added | jscs | If anything, @joran, it seems to me that the overall trend is more compelling, given that the number of >20k users strictly increases while the number of answers decreases. This is absolute answers, rather than proportion of all answers, so even if an individual answerer's rate drops as e crosses the 20k line, e should still be contributing to the group's remaining steady or even increasing. But I agree with you that the population change seems to be an important unconsidered factor (then again, I'm definitely not a statistician). | |
| Apr 29, 2014 at 19:43 | comment | added | joran | Thinking about this a bit more, I'd be interested in seeing the monthly proportion of answers coming from users with 20k+ rep at the time they answered the question. It is unsurprising to me that a common pattern would involve a sprint of activity to high rep followed by declining answer rate. But has the proportion of all answers coming from high rep users declined? I think that requires the ability to look backwards in time in rep, and I'm not sure the data extracts allow that. | |
| Apr 29, 2014 at 17:42 | comment | added | Gayot Fow | @A.Webb, put that as an answer and we can observe an informal benchmark on meta-sentiment | |
| Apr 29, 2014 at 17:38 | comment | added | Wooble | -1; needs more freehand red circles on the graphs. | |
| Apr 29, 2014 at 17:34 | comment | added | A. Webb | @GarryVass I would not assume that based on this data alone. I really would expect the sector economy to play a rather direct role. I took the Obama comment as facetious, especially since it was lumped in with the Mayan calendar, but I suppose there could be some health care reform effects -- I would be willing to bet one depreciated dollar that at least one high rep user is a "quitter" because they felt empowered to go into business for themselves without having to rely on their corporate insurance. | |
| Apr 29, 2014 at 17:34 | comment | added | joran | So, my first thought is that "high rep users" is not a static population. Are high rep users fungible? Are the declining contributions of current high rep users offset by the contributions of newly minted 20k users? That would probably require a somewhat different data set, though. | |
| Apr 29, 2014 at 17:21 | comment | added | Gayot Fow | @A.Webb, the NASDAQ and Obama cannot be discounted as indirect factors I suppose; but things closer to home are likely to have more a more direct influence | |
| Apr 29, 2014 at 17:19 | comment | added | David Robinson | @A.Webb: My apologies: I agreed entirely and was responding in kind (apparently to the point where the irony collapsed). Getting input from high-rep users whose activity was declined was indeed the reason I asked this question | |
| Apr 29, 2014 at 17:17 | comment | added | A. Webb | @DavidRobinson That comment was really just meant to counter the assumption that all the explanatory variables are specific to Stack Overflow, e.g. policy changes. The why probably can't be determined very well from the data. A user survey is needed to address that. | |
| Apr 29, 2014 at 17:10 | comment | added | A. Webb | If we are positing explanatory variables, I'd suggest also looking at a graph of the NASDAQ composite, which started surging at the beginning of 2013. Increase in technology sector means the top tier gets too busy to answer questions and that there are more newcomers and more questions. | |
| Apr 29, 2014 at 17:09 | comment | added | Niklas B. | Okay, I was thinking growth :) And there's a bit of a limit on how much you can write per day | |
| Apr 29, 2014 at 17:08 | comment | added | David Robinson |
@NiklasB.: No, the trend being fit is an exponential one, exp(intercept + rate * time) (specifically, using quasipoisson regression). This makes particular sense in terms of decay (a 5% decrease in answers per month)
|
|
| Apr 29, 2014 at 17:06 | comment | added | Niklas B. | @DavidRobinson Do you mean quadratic trend? | |
| Apr 29, 2014 at 17:04 | comment | added | David Robinson | @MartijnPieters: Ask and ye shall receive: imgur.com/C7SwS5s. Definitely a "Joiner" (no surprise there), though it doesn't perfectly fit the exponential trend. (Indeed, if anyone is a counterexample to your last comment, it's probably you) | |
| Apr 29, 2014 at 16:59 | comment | added | Martijn Pieters | I'd be curious to see what my profile looks like, btw. My experience with R is exactly 0, though. | |
| Apr 29, 2014 at 16:57 | comment | added | Gayot Fow | @DavidRobinson Indeed plausible. Your time series annotated with significant policy changes would be invaluable. From another angle, what about the RATE at which users are entering the high rep zone during the same period? I suspect that has slowed dramatically also? | |
| Apr 29, 2014 at 16:57 | comment | added | Martijn Pieters | Perhaps because the high-rep users no longer have to work so hard to gain rep? The backlog of answers created provides a steady 'income' of reputation. | |
| Apr 29, 2014 at 16:50 | comment | added | David Robinson | @GarryVass: It looks like that happened in January 2013, which was indeed the start of the sharp drop in answering. I can see how the connection is plausible, though of course the evidence is circumstantial | |
| Apr 29, 2014 at 16:48 | comment | added | Gayot Fow | Question: which part of the drop, if any, is coincident to the site's removal of the person's ask/accept ratio which happened in 2012? | |
| Apr 29, 2014 at 16:42 | comment | added | BoltClock Mod | I still answer questions regularly even amidst my mod duties, but even I am aware I'm not as prolific with answers now as I was in my first year/first 20k rep of activity. It's strange, because I have over 10 times as much rep as I gained in my first nigh-year of activity, but my rep gain hasn't actually accelerated let alone my level of activity. | |
| Apr 29, 2014 at 16:38 | history | answered | David Robinson | CC BY-SA 3.0 |
