Do theverge have this big font or is something broken on my end?
You can download the entirety of Wikipedia for offline usage, BTW. I do this with an application called Kiwix https://kiwix.org/en/ .
Click “All Files” on the left menu of the program.
In the bottom search bar (there is one top and one bottom bar) type “wikipedia” to show only those entries matching the search.
Then click on the “Size” header to sort all entries by size. Usually the biggest one is the most complete.
Now “Download” it (i already have it, so it says “Open” for me).
Note that the big one with 111 GB contains images and contains all English language Wikipedia articles. The one with 43 GB should be the same I think, but without images. There are many other variants too, varying in content and theme and even build date. In example the one with “1m Top” contains the top 1 million articles only.
It doesn’t actually include all the media, and – I think – edit history. It does give you a decent offline copy of the articles with at least the thumbnails of images though.
Edit: If you want all the media from Wikimedia Commons (which may also include files that are not in Wikipedia articles directly) the stats for that are:
Total file size for all 126,598,734 files: 745,450,666,761,889 bytes (677.98 TB).
To be clear, I’m fine with RAM being base 2 – it’s rather difficult for it not to be given the structure – but for fixed storage, this is an old-school measurement that only gets worse with each order of magnitude.
I don’t remember which is the stupid “1024 bytes in a kilobyte” one but
745,450,666,761,889 byte is 745 terabytes, that should be 745 TB and that 678 should be what TiB is for
And also that entire 677.98 is a useless value, there’s nothing that is “677” about this
It is if you just truncate! No one should do this, as I don’t recall the last time I saw such a textbook example of “rounding error” meaning “we fucked up while rounding.”
The problem with this solution is that it leaves out the most important part of Wikipedia of all; the editors. Wikipedia is a living document, constantly being updated and improved. Sure, you can preserve a fossil version of it. But if the site itself goes down then that fossil will lose value rapidly, and it’s not even going to be useful for creating a new live site because it doesn’t include the full history of articles (legally required under Wikipedia’s license) and won’t be the latest database dump from the moment that Wikipedia shut down.
Some solution is better than no solution. I don’t mind having a ‘fossil’ version for a pinch. We got along okay with hardcovered encyclopedias pre-internet and this is not that different except it still being reliant on electricity. (I have different, more valuable books on hand if we ever wind up THAT fucked.)
My point is that the alternative isn’t “no solution”, it’s “the much better database dump from Internet Archive or Wikimedia Foundation or wherever, the one that a new Wikipedia instance actually would be spun up from, not the one that you downloaded months ago and stashed in your closet.”
The fact that random people on the Internet have old copies of an incomplete, static copy of Wikipedia doesn’t really help anything. The real work that would go into bringing back Wikipedia would be creating the new hosting infrastructure capable of handling it, not trying to scrounge up a database to put on it.
Sure, but are any of these “don’t worry guys I torrented a database dump, it’s safe now” folks going to go to the trouble of actually doing that? They’re not even downloading a full backup, just the current version.
You need to devote a lot of bandwidth to keeping continuously up to date with Wikipedia. There’s only a few archives out there that are likely doing that, and of course Wikimedia Foundation and its international chapters themselves. Those are the ones who will provide the data needed to restart Wikipedia, if it actually comes to that.
Wikipedia is not at risk of being shutdown, the danger is malevolent editors bringing the culture war inside of it and destroying “truth”. While it would be great to keep wikipedia as it is, “they” are coming for it, wikipedia doesn’t get to be excluded from the war. For now the best we can hope for is that it will survive but the best we can do is save local wikipedia copies in case the worse happens. Which isn’t shutdown, but corruption.
Thanks for sharing this. Started hosting a local copy of several wiki sources last weekend once this news broke.
Another commenter said downloading is missing out on the best part of Wikipedia, the ongoing editing. Which, while true, is also going to be a weak point.
How many of those amazing editors are going to stick around when their full time job becomes combatting obvious right wing bullshit, when they have to submit gov ID to have an account on the site, and when common sense and fairness becomes a crime?
Wikipedia was a high point for humanity. Whatever comes next I’d like to preserve a little piece of it.
Honestly someone recently posted on the hisoricalness of jesus and the article seemed way different than a few years ago and I would say less accurate. Sorta wish I had downloaded it in like 2015.
Best thing is that it works flawlessly on the mobile apps as well, and Wikipedia also has a 1 million most relevant articles or so, which is just a few gigabytes.
Do theverge have this big font or is something broken on my end?
You can download the entirety of Wikipedia for offline usage, BTW. I do this with an application called Kiwix https://kiwix.org/en/ .
Note that the big one with 111 GB contains images and contains all English language Wikipedia articles. The one with 43 GB should be the same I think, but without images. There are many other variants too, varying in content and theme and even build date. In example the one with “1m Top” contains the top 1 million articles only.
The fact you can download the entirety of the site for 111gb sounds pretty damn impressive to me.
It doesn’t actually include all the media, and – I think – edit history. It does give you a decent offline copy of the articles with at least the thumbnails of images though.
Edit: If you want all the media from Wikimedia Commons (which may also include files that are not in Wikipedia articles directly) the stats for that are:
according to their media statistics page.
Nice stats. I always wondered. I get the feeling that ~678 TB is little bit more than ~111 GB.
Like, at least 7GB bigger.
We need a drive that’s at least… Three times this size!
Dear god, are we still using base 2 for file sizes? At least use TiB like a reasonable person.
It doesn’t matter in this case, as long as it is documented (and it is by the unit).
To be clear, I’m fine with RAM being base 2 – it’s rather difficult for it not to be given the structure – but for fixed storage, this is an old-school measurement that only gets worse with each order of magnitude.
deleted by creator
Yes, we all do
Nobody does that nerd
I don’t remember which is the stupid “1024 bytes in a kilobyte” one but
745,450,666,761,889 byte is 745 terabytes, that should be 745 TB and that 678 should be what TiB is for
And also that entire 677.98 is a useless value, there’s nothing that is “677” about this
It is if you just truncate! No one should do this, as I don’t recall the last time I saw such a textbook example of “rounding error” meaning “we fucked up while rounding.”
Text is light. Images are a bit heavier, but there’s not too too many.
The problem with this solution is that it leaves out the most important part of Wikipedia of all; the editors. Wikipedia is a living document, constantly being updated and improved. Sure, you can preserve a fossil version of it. But if the site itself goes down then that fossil will lose value rapidly, and it’s not even going to be useful for creating a new live site because it doesn’t include the full history of articles (legally required under Wikipedia’s license) and won’t be the latest database dump from the moment that Wikipedia shut down.
Some solution is better than no solution. I don’t mind having a ‘fossil’ version for a pinch. We got along okay with hardcovered encyclopedias pre-internet and this is not that different except it still being reliant on electricity. (I have different, more valuable books on hand if we ever wind up THAT fucked.)
My point is that the alternative isn’t “no solution”, it’s “the much better database dump from Internet Archive or Wikimedia Foundation or wherever, the one that a new Wikipedia instance actually would be spun up from, not the one that you downloaded months ago and stashed in your closet.”
The fact that random people on the Internet have old copies of an incomplete, static copy of Wikipedia doesn’t really help anything. The real work that would go into bringing back Wikipedia would be creating the new hosting infrastructure capable of handling it, not trying to scrounge up a database to put on it.
Isn’t there a way to sync the copy to the current version?
Sure, but are any of these “don’t worry guys I torrented a database dump, it’s safe now” folks going to go to the trouble of actually doing that? They’re not even downloading a full backup, just the current version.
You need to devote a lot of bandwidth to keeping continuously up to date with Wikipedia. There’s only a few archives out there that are likely doing that, and of course Wikimedia Foundation and its international chapters themselves. Those are the ones who will provide the data needed to restart Wikipedia, if it actually comes to that.
I don’t know but if there’s a way to get from WP only the history from a moment onwards, then it shouldn’t be that hard to update it.
Wikipedia is not at risk of being shutdown, the danger is malevolent editors bringing the culture war inside of it and destroying “truth”. While it would be great to keep wikipedia as it is, “they” are coming for it, wikipedia doesn’t get to be excluded from the war. For now the best we can hope for is that it will survive but the best we can do is save local wikipedia copies in case the worse happens. Which isn’t shutdown, but corruption.
Thanks for sharing this. Started hosting a local copy of several wiki sources last weekend once this news broke.
Another commenter said downloading is missing out on the best part of Wikipedia, the ongoing editing. Which, while true, is also going to be a weak point.
How many of those amazing editors are going to stick around when their full time job becomes combatting obvious right wing bullshit, when they have to submit gov ID to have an account on the site, and when common sense and fairness becomes a crime?
Wikipedia was a high point for humanity. Whatever comes next I’d like to preserve a little piece of it.
Honestly someone recently posted on the hisoricalness of jesus and the article seemed way different than a few years ago and I would say less accurate. Sorta wish I had downloaded it in like 2015.
Best thing is that it works flawlessly on the mobile apps as well, and Wikipedia also has a 1 million most relevant articles or so, which is just a few gigabytes.