Skip to main content

H. G. Wells: I believe that now and always the conscious selection of the best for reproduction will be impossible; that to propose it is to display a fundamental misunderstanding of what individuality implies. The way of nature has always been to slay the hindmost, and there is still no other way, unless we can prevent those who would become the hindmost being born. It is in the sterilization of failure, and not in the selection of successes for breeding, that the possibility of an improvement of the human stock lies. https://wordsmith.social/protestation/quotes#quote9236

H. G. Wells: The dominant men of the new time ... will find in themselves - it must be remembered I am speaking of a class that has naturally segregated, and not of men as a whole - a desire, a passion almost, to create and organize, to put in order, to get the maximum result from certain possibilities. They will be artists in reality, with a passion for simplicity and directness and an impatience of confusion and inefficiency. The determining frame of their ethics ... will be the elaboration of that future world state to which all things are pointing. ... It is manifest that a reconstructed ethical system ... will give very different values from those given by the existing system ... the ethical system of these men of the New Republic, the ethical system which will dominate the world state, will be shaped primarily to favour the procreation of what is fine and efficient and beautiful in humanity - beautiful and strong bodies, clear and powerful minds, and a growing body of knowledge - and to check the procreation of base and servile types, of fear-driven and cowardly souls, of all that is mean and ugly and bestial in the souls, bodies, or habits of men. To do the latter is to do the former; the two things are inseparable. In the new vision death is no inexplicable horror, no pointless terminal terror to the miseries of life, it is the end of all pain of life, the end of the bitterness of failure, the merciful obliteration of the weak and silly and pointless things. The new ethics will hold life to be a privilege and a responsibility ... and the alternative in right conduct between living fully, beautifully, and efficiently will be to die. For a multitude of contemptible and silly creatures, fear-driven and helpless and useless, unhappy or hatefully happy in the midst of squalid dishonour, feeble, ugly, inefficient, born of unrestrained lusts, and increasing and multiplying through sheer incontinence and stupidity, the men of the New Republic will have little pity and less benevolence. The men of the New Republic will not be squeamish, either, in facing or inflicting death, because they will have a fuller sense of the possibilities of life than we possess. They will have an ideal that will make killing worth while. The pre-eminent value of sexual questions in morality lies in the fact that the lives which will constitute the future are involved. If they are not involved, if we can dissociate this relationship from this issue, then sexual questions become of no more importance than the morality of one's deportment at chess, or the general morality of outdoor games. ... The men of the New Republic ... will rout out and illuminate urban rookeries and all places where the base can drift to multiply; they will contrive a land legislation that will keep the black, or yellow, or mean-white squatter on the move; ... so that childbearing shall cease to be a hopeful speculation for the unemployed poor; ... This thing, this euthanasia of the weak and sensual, is possible. On the principles that will probably animate the predominant classes of the new time, it will be permissible, and I have little or no doubt that in the future it will be planned and achieved. https://wordsmith.social/protestation/quotes#quote9237

George Bernard Shaw: The notion that persons should be safe from extermination as long as they do not commit willful murder, or levy war against the Crown, or kidnap, or throw vitriol, is not only to limit social responsibility unnecessarily, and to privilege the large range of intolerable misconduct that lies outside them, but to divert attention from the essential justification for extermination, which is always incorrigible social incompatibility and nothing else. https://wordsmith.social/protestation/quotes#quote9238

Short Planned Maintenance Tonight

My apologies if this is inconvenient, I opted to do it on shorter notice without a set hour because (a) there's not a lot of activity on the server and (b) I'm really impatient.

I'm doing a hardware upgrade that requires rebooting the network storage backend which will bring down everything for a short time. It should take well under 30 minutes to do the hardware swap and most of the downtime is just going to be the database starting back up (which often takes in the range of another 30 minutes).

As part of this I'll also be deploying some software updates that require a reboot to take effect.

WTF?

I honestly haven't the foggiest idea how this happened, but apparently the DNS settings got changed a few days ago on the servers with absolutely no explanation (and to junk nonsense settings for some reason). I'm going to keep an eye on them to make sure they don't change again.

Additionally I think that created a cascade that caused the other problems.

Any posts you've made over the past 2-3 days haven't been sent to other servers, but will start sending now.

As far as the other problems, I think when that happened it caused so many processes to lag and take way longer and more resources than usual as any time it tried to contact another server it timed out on the dns request.

DOS Overload

There's been some recent outages of the server, the root cause I've tracked down to the server getting overloaded with requests (mostly updates from other servers). Those updates have been coming in faster than the server can process them and preventing other requests from coming through.

I've made some tweaks that I believe have resolved it, fingers crossed.

Technical explanation:

The servers ran out of php-fpm threads to handle requests. It was configured with static count of 30 each (60 total). They were definitely impacted significantly by memory leaks which kept the count low.

I've changed it from static to ondemand and increased the count to 100 each, I'll probably go in and increase it again since it's still pegged at that limit almost constantly. But thankfully running on-demand seems to be keeping the memory usage per thread drastically lower.

Where the static assignment of 30 was eating up 8GB of ram, 100 on-demand threads is only taking up 1.3GB.

I'm going to increase it until it's either hitting memory constraints or it's no longer constantly at full capacity.

in reply to Server News

Server News
Server News
•

There's definitely some sort of time and code problem involved as it hit again this morning even with the previous changes, though this time it only impacted updates (making posts/comments/likes, getting new posts). I think reading was unaffected because those operations are faster and require significantly less memory.

For whatever reason, sometime around midnight the server gets hit with a bunch of requests that all seem to lock up, eating up large quantities of memory and then won't exit. (With on-demand the threads exit after 10s of being idle, there was over 100 threads running continuously from midnight until I killed them around 9am). Likewise there was a very massive flood of updates from other servers corresponding to that, so I think it might just be a bunch of large servers sending bulk updates or some such.

New tuning to handle that: I put firmer time limits into PHP to prevent threads from running forever, there's two options for setting max times and the first was getting ignored (I think friendica overrode it? the second should override that and kill any threads going too long)

In addition to that, I set up a rate limiter to the inbox endpoint (where other servers send updates to), this should help keep that from overloading the server (majority of the time it'll just be slowing them down by a second or two unless the server is overloaded, at which point the rate limit should help get it accessible for users)

⇧