Skip to main content



Edward Bernays: The engineering of consent is the very essence of the democratic process, the freedom to persuade and suggest. wordsmith.social/protestation/…


Lenin: Bourgeois society is continually producing politicians who love to assert they belong to no class, and opportunists who love to call themselves socialists, both of whom deliberately and systematically deceive the masses with the most florid and "radical" words. In times of crisis, however even well-meaning participants therein very often reveal a discrepancy between word and deed. The great and progressive significance of all crises, even the gravest, most arduous and painful, lies in the tremendous speed, force and clarity with which they expose and sweep aside rotten phrases, even if well meaning, and rotten institutions even if they are built on the best of intentions. wordsmith.social/protestation/…


Stalin: Leninism grew up and took shape under the conditions of imperialism, when the contradictions of capitalism had reached an extreme point, when the proletarian revolution had become an immediate practical question, when the old period of preparation of the working class for revolution had arrived at and passed into a new period, that of direct assault on capitalism. Lenin called imperialism "moribund capitalism." Why? Because imperialism carries the contradictions of capitalism to their last bounds, to the extreme limit, beyond which revolution begins. Of these contradictions, there are three which must be regarded as the most important. The first contradiction is the contradiction between labour and capital. Imperialism is the omnipotence of the monopolist trusts and syndicates, of the banks and the financial oligarchy, in the industrial countries. In the fight against this omnipotence, the customary methods of the working class-trade unions and cooperatives, parliamentary parties and the parliamentary struggle-have proved to be totally inadequate. Either place yourself at the mercy of capital, eke out a wretched existence as of old and sink lower and lower, or adopt a new weapon-this is the alternative imperialism puts before the vast masses of the proletariat. Imperialism brings the working class to revolution. The second contradiction is the contradiction among the various financial groups and imperialist Powers in their struggle for sources of raw materials, for foreign territory. Imperialism is the export of capital to the sources of raw materials, the frenzied struggle for monopolist possession of these sources, the struggle for a re-division of the already divided world, a struggle waged with particular fury by new financial groups and Powers seeking a "place in the sun" against the old groups and Powers, which cling tenaciously to what they have seized. This frenzied struggle among the various groups of capitalists is notable in that it includes as an inevitable element imperialist wars, wars for the annexation of foreign territory. This circumstance, in its turn, is notable in that it leads to the mutual weakening of the imperialists, to the weakening of the position of capitalism in general, to the acceleration of the advent of the proletarian revolution and to the practical necessity of this revolution. The third contradiction is the contradiction between the handful of ruling, "civilised" nations and the hundreds of millions of the colonial and dependent peoples of the world. Imperialism is the most barefaced exploitation and the most inhumane oppression of hundreds of millions of people inhabiting vast colonies and dependent countries. The purpose of this exploitation and of this oppression is to squeeze out super-profits. But in exploiting these countries imperialism is compelled to build these railways, factories and mills, industrial and commercial centers. The appearance of a class of proletarians, the emergence of a native intelligentsia, the awakening of national consciousness, the growth of the liberation movement-such are the inevitable results of this "policy." The growth of the revolutionary movement in all colonies and dependent countries without exception clearly testifies to this fact. This circumstance is of importance for the proletariat inasmuch as it saps radically the position of capitalism by converting the colonies and dependent countries from reserves of imperialism into reserves of the proletarian revolution. wordsmith.social/protestation/…


WTF?


I honestly haven't the foggiest idea how this happened, but apparently the DNS settings got changed a few days ago on the servers with absolutely no explanation (and to junk nonsense settings for some reason). I'm going to keep an eye on them to make sure they don't change again.

Additionally I think that created a cascade that caused the other problems.

Any posts you've made over the past 2-3 days haven't been sent to other servers, but will start sending now.

As far as the other problems, I think when that happened it caused so many processes to lag and take way longer and more resources than usual as any time it tried to contact another server it timed out on the dns request.



DOS Overload


There's been some recent outages of the server, the root cause I've tracked down to the server getting overloaded with requests (mostly updates from other servers). Those updates have been coming in faster than the server can process them and preventing other requests from coming through.

I've made some tweaks that I believe have resolved it, fingers crossed.

Technical explanation:

The servers ran out of php-fpm threads to handle requests. It was configured with static count of 30 each (60 total). They were definitely impacted significantly by memory leaks which kept the count low.

I've changed it from static to ondemand and increased the count to 100 each, I'll probably go in and increase it again since it's still pegged at that limit almost constantly. But thankfully running on-demand seems to be keeping the memory usage per thread drastically lower.

Where the static assignment of 30 was eating up 8GB of ram, 100 on-demand threads is only taking up 1.3GB.

I'm going to increase it until it's either hitting memory constraints or it's no longer constantly at full capacity.

in reply to Server News

There's definitely some sort of time and code problem involved as it hit again this morning even with the previous changes, though this time it only impacted updates (making posts/comments/likes, getting new posts). I think reading was unaffected because those operations are faster and require significantly less memory.

For whatever reason, sometime around midnight the server gets hit with a bunch of requests that all seem to lock up, eating up large quantities of memory and then won't exit. (With on-demand the threads exit after 10s of being idle, there was over 100 threads running continuously from midnight until I killed them around 9am). Likewise there was a very massive flood of updates from other servers corresponding to that, so I think it might just be a bunch of large servers sending bulk updates or some such.

New tuning to handle that: I put firmer time limits into PHP to prevent threads from running forever, there's two options for setting max times and the first was getting ignored (I think friendica overrode it? the second should override that and kill any threads going too long)

In addition to that, I set up a rate limiter to the inbox endpoint (where other servers send updates to), this should help keep that from overloading the server (majority of the time it'll just be slowing them down by a second or two unless the server is overloaded, at which point the rate limit should help get it accessible for users)