Denise (
denise) wrote in
dw_maintenance2026-05-31 10:00 pm
(no subject)
Robby has managed to put in a temporary fix for the site errors and things failing to refresh or not showing up where they should! The permanent fix is going to need Mark's experience, and unfortunately -- seriously, this literally never fails -- Mark has been on an international flight all day, because of course he has. (Never. Fails. He and I are not allowed to both take vacation at once.)
The site will work just fine with the temporary fix in place, things just might be a little slow here and there. We'll keep you updated.

no subject
no subject
This one's not code related!
no subject
But thank god, everything seems to be fine now.
Thank you so much for this network.
no subject
no subject
no subject
no subject
no subject
no subject
no subject
no subject
no subject
no subject
no subject
no subject
no subject
no subject
no subject
Thanks!
no subject
Thank you
--
А.Кац и Я
no subject
no subject
no subject
Yeah, my understanding of the technical details going by (that I am only half following, so this may not be the exact correct answer) is that one of the databases in one cluster got knocked over and stopped replicating in a very messy way that didn't properly take it out of the cluster, our redundancies kicked in exactly the way they're designed to and the other databases kept happily accepting and storing the data people were sending, but because the unhappy database wasn't out of the pool, the webservers were trying to hit the other databases in that failing-replication cluster and getting out of date information. Which, again I caution is just my understanding and could be wrong, but the temp fix last night was taking that entire failed-replication cluster out of the pool and that worked to solve the problem, and poor jetlagged Mark is now trying to debug the problem with that cluster to bring it back up.
no subject
no subject
no subject
When Mark told me he was going to be traveling for a few weeks I almost told him to take an extra hour just to check up on all the systems that make DW run and work without needing any attention 99.99999% of the time, because the second he got on the plane, something was going to fall over. I decided I didn't want to curse his trip! I SHOULD HAVE KNOWN.
If we're both going somewhere for a conference or whatever, we seriously do plan our travel so we're not both on an airplane at the same time. Things that have been working seamlessly for years KNOW when we are both unreachable.
no subject
no subject
Would, or has, DW ever considered a donation drive to assist with rising costs and additional dev needs? Similar to how AO3 does regular drives. I'm sure there's a chunk of us who would be interested in chipping in financially beyond supporting the site but purchasing paids/points
no subject
Because we're a for-profit company and not a nonprofit like AO3 is, we can't accept donations without providing corresponding services without the accountant and the IRS making increasingly unhappy noises at us, unfortunately. (I did an incredible amount of research and consulted a bunch of lawyers when we were getting started, to see if there were any way we could operate as a nonprofit, but a combination of federal and state laws around nonprofit funding, the way we were planning to restrict some features/higher limits to paid users for cost control purposes, and the amount of paperwork and overhead it would require, which my ADHD ass is absolutely terrible at, meant we couldn't find a way to do it. The OTW being structured differently and specifically having "nonprofit member" so divorced from "user of AO3" and not having AO3 features that are only available to paid accounts/donors, means they could do it a lot more easily than we could, especially since their economics of hosting, and especially "cost per active user" and "motivation to pay not being as tied to degree of usage", are radically different than ours due to the different features between the sites.)
We've needed to raise our prices for a bit, because they haven't changed since launch in 2009 and inflation has eaten away at the purchasing power of $35 a lot, but I've been hesitating to pull the trigger because times are tough for everyone and we are making it work, it's just a bit more hustle. (There's also the factor of raising prices meaning some people have to stop paying or have to pay for fewer accounts, which means that raising prices by $X doesn't mean you make "X times your current number of subscribers" more money; in some cases raising prices can mean you actually lose money. I don't think that will apply here, but I also can't predict what the loss-of-subscribers rate will be!)
In the meantime, if you do have the spare money and you want to support us more, which we would gratefully and wildly appreciate: the best way to "donate" to us is to buy a paid account for a random active free user! That gives us extra money, sidesteps the accounting problem, introduces someone who's active on the site to the paid account features, and is a random act of kindness that gives someone a small warm fuzzy feeling and the world needs more of those in general right now. You can also look up accounts in your circle who don't have paid accounts or whose paid accounts are expiring soon, and if all else fails, I know some people have a very sweet habit of sending paid time to people who comment on dw-news and dw-maintenance posts or using the site search to look for people who post about their paid time expiring soon and them not being able to afford to renew it and renewing it for them.
no subject
Off to gift a few paid accounts! :)
no subject
I would genuinely love if we could be a nonprofit, because it's the best way to signal that we are motivated by things other than "line go up forever" and that we are happy being a small, independent, user-funded service that cares a lot more about your privacy, anonymity, and security than about money (which I recognize is a very unusual stance and is hard for people to believe until they see the evidence!), but we just could not make it work. The paperwork and administrative overhead necessary being ADHD kryptonite is a problem we could have solved by throwing money at someone to do it for me, but the fundamental issues around funding model and provision-of-extra-services just were completely unworkable. By the time my lawyer friends were making the "strained grimace" face and sketching out a minimum of three registered entities and some extremely complex licensing and funding agreements, I said "okay, nevermind, we'll just build trust the old-fashioned way and hope people believe us about being very bad at capitalism", heh.
no subject
Never has this icon been more apropos.
no subject
no subject
More detail: I got a reply to my support request in the small hours (in my time zone, GMT+1), and tried this morning to access this post and the ones that failed yesterday; I got one more Oops error and a lot of timeouts. But this afternoon it's working smoothly, hurrah!
no subject
I think mobile posting unfortunately broke during this whole mess and remains broken - not posting comments or sending error messages.
I submitted a support request but am commenting here just in case anyone else lost some comments to the abyss without noticing and needs to copy+paste them from their sent mail folder.
no subject
When you say "mobile posting", do you mean post-by-email ie replying to comment notifications, or something else?
EDIT: never mind, I missed that you said "sent mail folder" in your comment! I think I know why that's acting up and it will take a bit to fix; I'll post another top level if it turns out I'm right.
Thank you very much
--
А.Кац и Я
no subject
no subject
no subject
no subject
Thanks for fixing this problem so quickly!
no subject
no subject
I get the following: Error contacting server: SyntaxError: Unexpected token '<', " But it goes away when I refresh, and the deleted comment stays deleted, as I wanted.
I keep looking at support tickets, and no one's reporting a similar issue, so I'm wondering if it's something that's somehow connected to my IP address, or if it's sitewide? It's something I can live with, so I didn't see a need to open a support ticket. It's just mildly irritating.
no subject
We first noticed that about a month ago and have had it logged as an issue! The frontend wizard thinks it should go away with the next code push, which we're aiming for "when Mark gets back from vacation" or thereabouts.
no subject
Also, I hope Mark enjoys his vacation; you people work hard keeping this place running, so I know it was earned!