Slashdot Mirror


Wikipedia Gears Up For Explosion In Digital Media

jbrodkin writes "Wikipedia is gearing up for an explosion in digital content with new servers and storage designed to handle larger photo and video uploads. Until early 2008, the user-generated encyclopedia's primary media file server had just 2TB of total space, which was not enough to hold growing amounts of video, audio and picture files, says CTO Brian Vibber. 'For a long time, we just did not have the capacity [to handle very large media files],' he says. Wikipedia has raised media storage from 2TB to 48TB and the limit on file uploads from 20MB to 100MB. Ultimately, Wikipedia wants to eliminate any practical size limits on uploads, potentially allowing users to post feature length, high-quality videos. 'The limits will get bigger and bigger to where it will be relatively easy for someone who has a legitimate need to upload a two-hour video of good quality,' Vibber says."

44 of 141 comments (clear)

  1. I can only imagine how bad the edit wars will be by jollyreaper · · Score: 3, Interesting

    The Wiki project represents the best and worst that's in us. I wonder if people will start trying to archive classic shows on there like they do on youtube. :)

    --
    Kwisatz Haderach
    Sell the spice to CHOAM
    This Mahdi took Shaddam's Throne
  2. Youtube? by Anthony_Cargile · · Score: 4, Interesting

    Why don't they instead just allow linking to youtube videos without the WP nazis removing them? Sure they can upgrade storage size, but if they start storing videos everyone wants to see, then you're looking at youtube-sized bandwidth bills (or lack thereof) ensuing. It makes more sense to me, at least. [citation needed]

    1. Re:Youtube? by A+nonymous+Coward · · Score: 5, Insightful

      And youtube may withdraw them or restrict their audience at any time.

    2. Re:Youtube? by Eil · · Score: 4, Insightful

      Why don't they instead just allow linking to youtube videos without the WP nazis removing them?

      First, presumably the article probably means Wikimedia Commons rather than Wikipedia itself. That said, one of Wikipedia's biggest goals is to have all media content as open and accessible as possible. They accept only free, open, and unencumbered file formats.

      YouTube is pretty much the exact opposite of Wikipedia. That is, you cannot download the content for your own use or to redistribute it, there is no open source software that can easily view YouTube content, there is no intelligent discussion of said content (only "omfg americas r soooo dumb"), and nobody except YouTube employees are allowed to express an opinion on whether or not the content is suitable for deletion. And finally, there is no certification that the content being viewed is in the public domain or is being used within the bounds of fair use.

  3. Wikipedia=new on-line data repository by prgrmr · · Score: 3, Funny

    Do you have a large video but don't want to consume your desktop hard drive with it? Just write an article about it and post it all to Wikipedia.

    1. Re:Wikipedia=new on-line data repository by Anonymous Coward · · Score: 3, Insightful

      And it'll get speedy-deleted on grounds of notability, original research, etc - and you won't have a video anymore.

    2. Re:Wikipedia=new on-line data repository by Abstrackt · · Score: 3, Funny

      [citation needed]

      --
      They say a little knowledge is a dangerous thing, but it's not one half so bad as a lot of ignorance. - Terry Pratchett
    3. Re:Wikipedia=new on-line data repository by TubeSteak · · Score: 2, Interesting

      And it'll get speedy-deleted on grounds of notability, original research, etc - and you won't have a video anymore.

      Does anything on Wikipedia ever really get deleted?
      I thought the Mods and Admins had full access to deleted pages.

      --
      [Fuck Beta]
      o0t!
    4. Re:Wikipedia=new on-line data repository by Simetrical · · Score: 2, Insightful

      Does anything on Wikipedia ever really get deleted? I thought the Mods and Admins had full access to deleted pages.

      Yep, that's generally true. Anyone who can delete things can also undelete things, and there are lots of people who can do both: over 1600 on the English Wikipedia, 250 on the Wikimedia Commons -- any administrator. Hypothetically a sysop would be able to use Wikipedia as a private file store this way, since views of deleted content aren't logged, but that's probably not worth it. :)

      If you upload something that even the admins shouldn't see, generally an "OMG lawsuit" kind of thing like personal information, you can get your revision oversighted -- still stored, but only restorable by someone with shell access. This doesn't currently work for uploads, though, as far as I know.

      Actually, though, deletion of files was permanent for a long time, until a couple of years ago. This created a fun doomsday scenario where a rogue or compromised sysop account could run a script to delete all images on Wikipedia unrecoverably. I don't think backups were kept then either, so they'd have to be manually gotten back from mirrors and things like that. Fun stuff. Part of the new hardware setup uses ZFS snapshots to back up the files now, from what I've been told, although I haven't worked with that directly.

      --
      MediaWiki developer, Total War Center sysadmin
  4. Hahaha. Typical. by Anonymous Coward · · Score: 2, Insightful

    Wikipedia, as a nonprofit, is no different from any other dumbass venture-backed company.

    "Hey we just collected $6MM, and we're heading into Great Depression II. What should we do?"

    "Why don't we spend all of it as quickly as we can, then beg for more in a few months?"

    "Genius! Give that man a raise!"

  5. Re:One word by jellomizer · · Score: 2, Informative

    BitTorrent doesn't work well with unpopular information.

    --
    If something is so important that you feel the need to post it on the internet... It probably isn't that important.
  6. Re:I can only imagine how bad the edit wars will b by truthsearch · · Score: 3, Insightful

    I doubt it, due to copyrights. The expiration on copyright is so long that they'd have little to legally archive.

  7. Wikipedia = The Internet by neoform · · Score: 5, Insightful

    Seriously, half the pages I view on a daily basis these days are wikipedia pages. Any time I want to learn about something, it's the first place I go.

    --
    MABASPLOOM!
    1. Re:Wikipedia = The Internet by geobeck · · Score: 4, Insightful

      Any time I want to learn about something, it's the first place I go.

      I sincerely hope it's not the last place you go.

      That's the key. I agree with the previous poster; Wikipedia is a great place to start your online research. But of course I never quote the Wikipedia article itself (except for minor things like atomic weights and other easily-verifiable data). A well-written Wikipedia article is a speedy link to a collection of journals, newspaper articles, and primary sources.

      Conversely, of course, a poorly-written Wikipedia article is a speedy link to a collection of 'authoritative' blogs, home pages and fringe websites.

      Wikipedia is a great research tool for anyone who knows how to perform research.

      --
      Find environmentally and socially responsible products on http://buy-right.net
    2. Re:Wikipedia = The Internet by quintessentialk · · Score: 2, Insightful

      Here's a test. Pick a subject that you are expert in, or even have a good passing knowledge of -- any subject, pick a few even. Go to the wikipedia page on that topic, and you will find inconsistencies, inaccuracies, conjecture, missing information and sometimes downright lies.

      I've found Wikipedia to be very accurate on topics in mathematics, physics, basic chemisry, and other 'nerdy but not controversial' topics (especially as a general reference for formulas, constants, and methods). When I've examined articles on topics about which I'm especially familiar I've found that writing quality and organization are pretty good indicators of accuracy. I assume that applies broadly.

      That's not the point, though. You're absolutely right that wikipedia shouldn't be the final source for anything critically important (with few exceptions). But it is good enough for most casual (entertainment) tasks, and even many professional ones, assuming you work with hard sciences.

  8. legitimate need? by nurb432 · · Score: 3, Insightful

    "legitimate need to upload a two-hour video of good quality"

    Who gets to define legitimate?

    --
    ---- Booth was a patriot ----
    1. Re:legitimate need? by JustinOpinion · · Score: 3, Insightful

      The same people who determine whether a given paragraph legitimately deserves to remain in a Wikipedia article: the community of volunteer contributors.

      For better or worse, the people deciding what videos should be kept and which should be deleted will be those who are involved and passionate about Wikipedia. If you think Wikipedia is doing overall a good job so far, then presumably you expect them to make good decisions about what videos are worthwhile. If you think Wikipedia is overall doing a poor job, then presumably you expect them to make poor and/or capricious choices with respect to video.

    2. Re:legitimate need? by Simetrical · · Score: 2, Informative

      Actually, most of these uploads should go to the Wikimedia Commons, not Wikipedia proper. Files uploaded to Commons can be used on any Wikimedia site, including any language of Wikipedia, Wiktionary, etc. Files uploaded to the English Wikipedia can only be used on the English Wikipedia. The Commons admins are largely a different group of people from the English Wikipedia admins, although there's some overlap. Adminship is given out on a per-project basis; only a few dozen stewards have any privileges across projects.

      --
      MediaWiki developer, Total War Center sysadmin
  9. Typo by brian0918 · · Score: 2, Insightful

    Unlike my name, Vibber's is spelled BriOn.

  10. Re:Whatever happened with google sponsorship? by Anonymous Coward · · Score: 2, Informative

    No, just a rumour. Google never sponsored anything about Wikimedia other than the occasional party at the annual conference. Yahoo!, on the other hand, has been hosting a Wikipedia data room within their data centre in Seoul since 2004/5-ish. Just goes to show how inaccurate these Interweb thingies are. :-)

  11. I forsee a new job at Wikipedia... by GPLDAN · · Score: 4, Insightful

    ...video editor

    Am I going to the Wikipedia page on France, and watching a video, complete with caption in *My* language, of France - like a mini-documentary or travel brochure or promo? Who produces that? Who edits it? Is there a standard narrator? Can we get that guy with the cool voice that does Frontline to do them all? Will they have any standards in how they are produced? How they are credited?

    There is a fundamental and critical difference between Youtube, which is a Bazaar, and Wikipedia, which is a Cathedral - to brazenly steal Eric Raymond's title.

    A video on say France is the authoritative video on the subject. Unlike say a picture, which may be used or copied with permission that may show a city or a map, videos require much more work. Will Oliver Stone get to do the video for George W Bush? Will it be like the BMW series with Clive Owen, having a bunch of guest directors? Can we have Marty Scorsese do the video for New York City?

    Multimedia is cool, but it opens up alot of problems.

  12. Commons? by JeffSh · · Score: 2, Informative

    Sounds pretty dumb to me. Media should be at Wikimedia commons, not in Wikipedia proper.

    Maybe that's what he means, but I didn't RTFA.

  13. Re:I can only imagine how bad the edit wars will b by dedazo · · Score: 3, Insightful

    Their main problem is going to be making sure that none of the stuff people upload violates any copyright and conforms to their free/non-free usage guidelines. There are only so many user-generated videos that could find a place in an encyclopedia, so I assume most of what they'll see will be ripped from other places.

    They spend enormous amounts of time "patrolling" uploaded images, placing them on special categories for later review and so on. And the processes in place don't help, either. The last time I tried the upload page for an image from the Cassini mission I was pretty much blown away how complicated it is to figure out how to tag a file to avoid having it be deleted on sight, even though the use permissions from the copyright owner were pretty clear.

    If the Wikipedia bureaucracy is bad now, just wait for the Video Upload Patrol Group to form up. Oh the humanity.

    --
    Web2.0: I love when people Flickr my cuil and digg my boingboing until my google is reddit and I start to yahoo
  14. Re:I can only imagine how bad the edit wars will b by Hatta · · Score: 2, Insightful

    We already have archive.org for anything out of copyright, or freely redistributable. There are even full length features available.

    My question is how exactly is a 2 hour movie going to fit in with the mission of Wikipedia. They're intended to be an encyclopedia, not a movie download service. It would make sense to link to clips of films in the article on John Williams or Spike Lee or whoever, but all you need is a clip, not the whole film.

    --
    Give me Classic Slashdot or give me death!
  15. A chance for .ogg to shine by bogaboga · · Score: 3, Interesting

    These developments offer a chance for the open source .ogg/theora format to shine.

    While folks at Illiminable have done a good job of providing a codec to play .ogg files within Windows Media Player, I hope this can be available by default.

    That is, you attempt to play an .ogg/theora file and the system provides a opportunity to download and install/setup the plugin by default on systems without the ability to play .ogg/theora files.

    1. Re:A chance for .ogg to shine by LWATCDR · · Score: 4, Insightful

      Except that theora isn't all that good. Yes it is free but the quality isn't as good as many other codecs out there. I wish that Dirac would get more attention as a codec.

      --
      See my blog http://ilovecookes.blogspot.com/ for light hearted technical information.
  16. Re:I can only imagine how bad the edit wars will b by MightyMartian · · Score: 4, Funny

    I eagerly await the update to this: http://en.wikipedia.org/wiki/Sexual_positions

    --
    The world's burning. Moped Jesus spotted on I50. Details at 11.
  17. Too much of a burden on Wikipedia by MobyDisk · · Score: 5, Insightful

    This is out of scope for Wikipedia. It sounds like this should be an entirely separate project. Wikipedia is an encyclopedia. Encyclopedias should not have video:

    I don't mean that because traditional encyclopedias did not have video, but because it doesn't fit with the type of content that an encyclopedia presents. It is similar to how newspapers should not have video. Wikipedia is not a teaching tool. It is not meant to provide functional examples. It is a starting point: a dictionary-style explanatory description.

    An entry on the Hindenburg does not need a video of the Hindenburg disaster. It needs technical specifications, historically accurate statements of what happened, and a link to a museum who DOES house the video.

    An entry on Calculus needs a historic description and a mathematical overview. Not a 2-hour lecture.

    Now --- that doesn't mean that a video repository is not a good project. I think that would be awesome. Youtube kinda has that, but it has garbage thrown in. But maybe Wikipedia is not the place for it.

    1. Re:Too much of a burden on Wikipedia by JPortal · · Score: 5, Insightful

      I don't see why that's outside the scope of Wikipedia. A video of the disaster could fall under "historically accurate [depiction] of what happened."
      I agree that lectures would be a bad idea, but some full-length videos are very informative and useful for research purposes.

    2. Re:Too much of a burden on Wikipedia by jubei · · Score: 2, Insightful

      Videos can be especially helpful when used to describe motions and processes. Things like engine cycles, swarming behavior, and traffic patterns would be good subjects for videos.

    3. Re:Too much of a burden on Wikipedia by Thaelon · · Score: 2, Interesting

      An entry on the Hindenburg does not need a video of the Hindenburg disaster.

      I disagree. I think that's a perfect example of an article that needs video. In my mind the video starts when something goes visibly wrong to the point that it's a pile of stationary, yet flaming wreckage on the ground and that's it.

      Contrast this with typical American TV that is so fucking full of filler like commentary and "dramatic" camera movements that I can't watch it anymore. Like "World's Most Dangerous Police Chases" and the like. There's like 3 minutes of worthy video padded with 3-10 repetitions and 21 minutes of inane blathering no one cares about. The kind of stuff that is invariably absent on youtube renditions - even those taken from the show.

      I think there are exceptional cases where a video is warranted, but they should be extremely short. No commentary whatsoever. Text is a better format for it.

      This turned out to be a kind of exploratory essay, and I apologize, but I guess the conclusion is: Yes wikipedia should have video, but only in exceptional cases, and keep it as short as possible, and no speaking.

      Lastly, I expect Wikipedia's video posting rules to cite this post.*

      *Warning: do not attempt to read this sentence without a sense of humor.

      --

      Question everything

    4. Re:Too much of a burden on Wikipedia by Simetrical · · Score: 2, Informative

      Wikipedia is not the only Wikimedia Foundation project. In particular, the scope of the Wikimedia Commons is "to provide a media file repository . . . that makes available public domain and freely-licensed educational media content to all . . .". All the projects are run from the same servers, and share the same upload servers in particular -- notice how all uploaded images are at upload.wikimedia.org, no matter what the project is. The technical upgrades are of most value to Commons, which has long had trouble accepting in-scope content like high-quality, free educational videos because they're over the file size.

      --
      MediaWiki developer, Total War Center sysadmin
  18. This is going to be what YouTube was supposed tobe by DelgadoRandom · · Score: 2, Interesting

    Instead of just wading through a billion "OMG Kid FAILS at Suzuki ghost-ride in front of hott!!!" videos mismarked as news, people can actually get to the most representative and informative feeds on emerging issues. Like a public newsroom.

  19. Re:I can only imagine how bad the edit wars will b by AKAImBatman · · Score: 4, Interesting

    That doesn't prevent there from being a rather significant pool of classic media. Take the old Superman cartoons as an example. They all fell into public domain long before they could be grandfathered back into existence. Thus just about anyone who wants to host them, edit them, use them in a new work, or otherwise make use of those old films is able to do so. Also, some of those films are likely to be new works that are gifted into the Creative Commons in the same way the Wikipedia article text is. Think of a shark in its natural environment, a tour of a famous building, or even a re-enactment of a historical battle.

    There's even work that's been done to show how Wikipedia might use the HTML5 tag if and when it becomes widely deployed. (See this page for a dev version of Opera and 2 example Wikipedia pages that support & fallback content.) Despite the seeming incongruity of allowing videos inside Wikipedia pages, the demos shown is actually quite natural.

  20. Who will pay for this ? by eulernet · · Score: 4, Insightful

    If they increase the storage, it means that the traffic will explode.
    Who will pay for the bandwidth ?
    This year, it was 6 millions of dollars, but with videos, at least 10 times this amount will be needed.
    Does this mean that ads will appear ?

  21. Re:I can only imagine how bad the edit wars will b by legirons · · Score: 2, Informative

    We already have archive.org for anything out of copyright, or freely redistributable.

    Not for long - The Internet Watch Foundation have just blocked archive.org to all UK population.

  22. Wikipedia Search = Sucky by Frosty+Piss · · Score: 5, Insightful

    Any time I want to learn about something, it's the first place I go.

    It's the second place I go, because the Wikipedia Search "feature" sucks unless you know exactly what you're looking for. If only Wikipedia would either fix their broken "search" or simply integrate Google search into it?

    --
    If you want news from today, you have to come back tomorrow.
    1. Re:Wikipedia Search = Sucky by Cowmonaut · · Score: 2, Insightful

      How about you just google it in the first place? Chances are the wiki entry is the top search result ANYWAYS. And really, their search isn't bad. You just have to know what the subject is called and be willing to scroll down to see the "possible matches". The article your looking for is probably in the top 5....

    2. Re:Wikipedia Search = Sucky by Simetrical · · Score: 3, Informative

      There have been major improvements to search lately, thanks pretty much solely to the volunteer work of Robert Stojnic (rainman). You might want to try it out again. Still probably not quite up to Google levels in some ways, given the difference in budget of some billions of dollars versus ~$0, but it has better relevance than before and a lot more nice features now (e.g., "did you mean").

      --
      MediaWiki developer, Total War Center sysadmin
  23. How about offering images for download again? by harmonica · · Score: 2, Interesting

    It's been 18 months since Wikipedia provided bulk downloads of image data. That may not be a priority for most people, but offering everything for download is essential for an open project in my opinion. Add all new images of a month to YYYYMM.tar and offer that as a torrent.

  24. Re:I can only imagine how bad the edit wars will b by Hogwash+McFly · · Score: 2, Interesting

    Why the focus on 'movies'? There are many situations in which an article about a particular subject could be improved through the use of a high-quality, feature-length educational video. Wouldn't the article for the Amen Break be more interesting if this video appeared on the page, right there in the sidebar? To borrow your example, you wouldn't have a Spike Lee film, but a documentary about him, fleshing out the details in the article and offering insight that text alone can't provide. If a picture is worth a thousand words, then a moving picture must be worth millions.

    Of course, you open up a whole can of worms in the editing battle side of things. Tug-of-wars over text has proved bad enough, let alone people arguing over weasel words and unsupported claims in a thirty minute documentary.

    --
    Mother, do you think they'll like this sig?
  25. Re:I can only imagine how bad the edit wars will b by Gerzel · · Score: 2

    Long bought and paid for bribed long...

  26. Re:Wikipedia Search = Much better by Taxman415a · · Score: 3, Informative

    It used to be really bad I agree, and so would everyone else. But it has recently gotten much better. It was just never seen as a high enough priority given the shoestring budgets and other fires that needed to be put out.

    Now however it gives reasonable suggestions for misspellings and has better accuracy.

  27. Wikipedia doesn't need this. by Animats · · Score: 2, Interesting

    I don't see a role for Wikipedia in this. Archive.org already accepts video uploads of useful archival material, so that's covered. Wikipedia has enough trouble finding redistributable still images for articles. Who's going to create useful video for Wikipedia that isn't original research or a copyright violation?