Perl Migrates To the Git Version Control System
On Elpeleg writes "The Perl Foundation has announced they are switching their version control systems to git. According to the announcement, Perl 5 migration to git would allow the language development team to take advantage of git's extensive offline and distributed version support. Git is open source and readily available to all Perl developers. Among other advantages, the announcement notes that git simplifies commits, producing fewer administrative overheads for integrating contributions. Git's change analysis tools are also singled out for praise. The transformation from Perforce to git apparently took over a year. Sam Vilain of Catalyst IT 'spent more than a year building custom tools to transform 21 years of Perl history into the first ever unified repository of every single change to Perl.' The git repository incorporates historic snapshot releases and patch sets, which is frankly both cool and historically pleasing. Some of the patch sets were apparently recovered from old hard drives, notching up the geek satisfaction factor even more. Developers can download a copy of the current Perl 5 repository directly from the perl.org site, where the source is hosted."
There are significant advantages of Git over Subversion. RTFS for some.
Just to add insult to injury -- often, a Git checkout, which includes all history, takes up less space than a Subversion checkout for the same project, which doesn't even include recent commit log messages.
But think about this -- you're saying they should use a big, slow, central server, as a single point of failure, crippling offline development, complicating branches (especially merges), and several orders of magnitude slower for just about every operation, just so you don't have to learn a "weird" tool?
Don't thank God, thank a doctor!
There are also advantages to Subversion that Linus states himself [1]. Really the only one of note is that Git isn't so great at having multiple projects in the one repository and the recommendation is to have one per repository and have a super-project that contains pointers to others - which isn't so great a solution.
[1] It was stated in relation to the layout of the KDE repository: http://git.or.cz/gitwiki/LinusTalk200705Transcript
These distributed models work best if it's a large team, which potentially has more than one level of hierarchical structure.
You do typically have a canonical central repository managed by the project lead (in the Linux kernel's case, Linus's tree). But then sub-section leads might have their own canonical repository for that sub-section, and merge in their team members' changes into a stable state that they approve of before asking for those changes to get merged into the central branch. Or they might bundle up some particularly important set of changes for early merging "upstream", making sure they cleanly apply against the current central repo. That's all a nightmare to manage in SVN, which conceives of branches as something you do occasionally and keep around for a while, not as a hierarchical project-management tool.
On the other hand, if you have a relatively small or flat team, or one where the sub-sections break down really cleanly so each one can have its own central repo, it might not buy you much. I'm working on a small project with 4 people at the moment, and SVN is perfectly fine, and I can't really imagine what I'd do with a distributed version control system (I'd just use it like a centralized one, pushing everything to the one repo everyone pulls from).
10 PRINT CHR$(205.5+RND(1)); : GOTO 10
DVCS does not mean anti-centralized. DVCS does not introduce arguments between developers, rather ameliorating them as it's easier to try things out and becoming more knowledgeable before discussing issues. It's about how to define the build and release systems. Obviously, you need a 'head revision' or 'release branch' or whatever you want to name the code that's defined as the one version that makes up the product. Having input from different places makes no difference on the release part of the process. Developers move the changes to the release/central build version just like they would with the old model. Almost all resistance I've seen so far is something similar to 'I don't like this because I have to learn something new' obfuscated behind a bunch of misconceptions.
If you just want Perl6 you can use it today with Pugs. That is the "release often" and it finished.
If you want a real release candidate the problem is Parrot. Perl6 attempted some very complex stuff with the runtime, that so far has been challenging to implement. There is no guarantee these problems will get solved.
You can use Perl 6 right now if you want. It is available, just not rated for production yet. I can understand that probably not many folks will want to use it for real world purposes, but this is pretty far from at least my definition of vapor.
If the original perl git packing was done aggressively (which it definitely has been done), then doing an untuned repack can only make it worse.
IOW, the original perl git pack (that you got by cloning a well-packed repository) was probably done with a much larger search window for good delta candidates, and quite likely with a deeper delta chain than the default too. When you did your own aggressive repack, you undid all of that.
End result: normal developers should likely never use "git gc --aggressive". It's for the maintainer, who can also tune the various knobs, and who probably has a beefy machine that can afford to go the extra mile of using lots of CPU time to find an "optimal" pack.
takes up less space than a Subversion checkout for the same project
Only if you actually *want* the whole project. If on the other side you just want a single file or subdirectory, say in a large gigabyte size repo with graphics, textures and stuff, you have kind of a problem with git. A git clone always downloads the whole thing, svn on the other side allows you do download just what you want, so the download with SVN can easily become a few orders of magnitude smaller then with git.
git could really use a way to do shallow clones so that only those pieces get downloaded that are actually needed, git-clone --depth is a start, but not quite enough.
isn't centralization the heart of source code management
Not necessarily. Consider a common case:
- Project A works on a significant feature (say a new file system)
- As part of their work, they significantly restructure some related part (say how a fs ties into the kernel) and update other parts of the source to match
- Project B works on a different feature (say overhauling the interrupt thread implementation on SMP systems)
- Project B wants to update the same related parts (say how kernel modules, including file systems, tie into the kernel)
- Project A is already done, and is scheduled to get on the train before project B, so it's natural for B to integrate portions of project A and then track project A's fixes and updates to these portions up to when A integrates into the trunk
This situation is handled very cleanly by distributed systems like git and teamworks. As B selectively merges parts of A it picks up the change log. With svn when you diffpatch across from the A branch to the B branch you lose changelogs, there will be no record that these changes came from A but they'll appear independently in B. When A integrates to trunk if B simply tracks these it will appear as if B edited trunk. This is an inaccurate history. With p4 I get a headache just thinking about it.
s/is already done/is already done with the shared part/g
The joke is that git depends on perl.
We use it at work and it works much better than SVN did.
Apart from everybody's local copies, we keep a repository sitting on a central server. That repo's "master" branch is our release code and, since I'm responsible for the final product, I'm responsible for this branch. Our workflow is fairly simple:
Note that pushing to master doesn't break anybody else, ever, until they decide they're ready to deal with integrating their patch. Nobody ever does the, "Are you gonna commit first or should I?" thing anymore. Developers that are collaborating on a patch sync via a branch on the central server, or directly to each other's machines, or via emailed patches, whatever they want to do. Git doesn't care and neither do I.
It sounds like a lot of tedious work, but Git is just stupid-fast. In the common case the whole update master, rebase, cleanup commits, push cycle takes about as long as SVN used to take to update and then scan for changes and actually commit anything. In the uncommon case where there's a non-trivial merge, the merges tend to come out a lot cleaner since Git is trying to make your changes to the new master one commit at a time, rather than dumping all of the changes in master on top of your stuff (though it can also do that, if you happen to enjoy pain).
And while I prefer the manual approval approach (which scales by appointing trusted lieutenants to take over some of the work) since it keeps me in the loop and keeps everyone else honest, there's no reason you couldn't automate it. Some projects give everyone push access, but disallow anything but fast-forward (trivial nothing-to-merge) pushes to the central server, others I've heard of have people push to a staging branch and a bot on the server grabs the code, runs the test suite, and merges it if it's good. Access is ssh-based, and there are hooks all over the place, you can set up all sorts of schemes when it comes to control of the canonical central repo.
The thing we've found is that because we've all
git makes branching and merging easy enough that the question of "where is the central line?" isn't really an issue- developers can easily work on their own branches without worrying about other branches, and you can still push your developer branch to the central repository so that the question of "Where is this change? Is it in Steve's branch? Do I need to connect to his repository?" is also not an issue- Steve's branch can easily be in the central repository, Steve just needs to push changes in, just like he'd normally need to commit changes. Git's primary difference there is that "Steve's repository" is pretty much just a robust staging area for changes.
However, if you're used to centralized version control, you may miss things switching to git:
- Pick whether you want all or nothing in advance. You can either have "shallow" checkouts, which leave you with a crippled, broken, and useless copy that has no access to history functions, or you can have every change ever made. Once you've made this choice, you can only change your mind by cloning again. There is no way to gracefully get history as it is required.
- This means: no partial checkouts. This is a problem if you're used to versioning large binary files, or have large files which you won't care about for anything other than auditing reasons after a certain time.
- Which also implies: no "modules". This is a problem if you have lots of small related projects, which together make up one massive pool of code. You can have one massive project which everyone uses all of, or you can choose not to track the origin of files which you copy from one project to another. Having a "common" project shared by several others is not possible.
- Unless you try the "submodule" support, which is a broken hack that can devour changes far too easily to trust it to end-users. And submodule support does NOT allow copies from one "submodule" to another, or to your main project. Not while retaining history, anyway.
This is really all one flaw, re-stated five times. Fix this and git will be able to replace any centralized system. Without the change, I can't recommend it to anyone who is involved in a centralized project- at least not when there is a reason for being centralized.
Git is, despite proponent's claims, great for small projects which don't actually need to talk to anyone else and don't need to interface with any other projects. If your project involves other "projects" where the line between one and another is the least bit blurry, avoid git.
-- 'The' Lord and Master Bitman On High, Master Of All
The main things I like about git are its raw speed, its ubiquity (everyone and their dog has a git tutorial), and how simple its primitives are.
That is: I actually started with bzr, and I found that while there were some things that were much easier (bzr uncommit comes to mind), it's a lot easier to actually understand Git under the hood, in case I need to do some deep surgery on some history, say.
Then, too, there just seem to be more tools available -- gitk, git-cherry-pick, even git-stash, are things I don't remember from bzr or hg, but it's been awhile.
I see the point about Python, and I'm absolutely with you there. The reason it's not an issue is, I can't ever remember having to dig into Git source -- the most I might have to do is write a shell-script frontend to some of the tools already there. It's actually somewhat UNIX-y that way -- when was the last time you had to dig into the source of fileutils?
What's more, there are language bindings -- personally, I've used grit in Ruby. Easier than trying to talk to SVN with its XML output.
The main advantage of bzr, by the way, is its handling of directories -- it actually records renames. Git tries to detect them, but it works at the level of file contents, not directory structure -- so, for example, it'll detect if you move a function from one file to another, but it might have trouble if you rename a directory. For example:
mv a b
mkdir a
And, in another branch:
touch a/c
When they merge, where should c go? Git would probably put it in a, since the file's name is 'a/c'. Bzr would probably put it in b, since a was renamed to b, unless the same rename somehow made it into that other branch.
There is another reason I didn't mention -- I use Ruby. I believe Ruby itself is done on SVN, but it seems that every Ruby project ever has moved from SVN to Git, and most of them to Github. And it's just awesome to work with Github projects.
Don't thank God, thank a doctor!
You forget that Q also appeared in ST:V a few times.
2009: The Year Of The Truly Helpful Slashdot Grammar Nazi Watchmen
Consider yourself spoken to.
http://code.google.com/p/tortoisegit/
Roughly
Linus was very resistant to version control at all and could always find a reason (or excuse) not to use each version control system that came along.
Eventually someone decided to listen to every demand from linus and create a vcs that met all of them. The catch was it was not FOSS and the gratis version had some pretty obnoxious terms. Things reached a head after someone at OSDL reverse engineered the protocol and linus was basically forced to either scrap bitkeeper or quit his job at OSDL.
However the period with bitkeeper had convinced linus that version control was a good idea. But all the alternatives he could find were either too centralised or too slow. So he hacked together git.
note: i'm known as plugwash most places but i screwd up registering that here somehow in the past and now can't register
- This means: no partial checkouts. This is a problem if you're used to versioning large binary files, or have large files which you won't care about for anything other than auditing reasons after a certain time.
Yeah, I agree with this. Git blazes along with the largest of code trees and keeps its history data nice and compact, but it's not so great for binaries. We use git for code and SVN for all our binary data, which is optimal for us as the binaries are unmergeable anyway and we've got workflows in place to deal with that issue.
Having a "common" project shared by several others is not possible.
I'm not sure what the scenario you're thinking of is, submodules have solved this for us.
- Unless you try the "submodule" support, which is a broken hack that can devour changes far too easily to trust it to end-users. And submodule support does NOT allow copies from one "submodule" to another, or to your main project. Not while retaining history, anyway.
Can you elaborate on this? We use submodules at work and we've never had any problem with them (short of the odd time someone forgets to git submodule update, but the lines in the git status output usually clue people in). What do you mean copying from a submodule to another? You mean like copying between repositories in general, which SVN externals couldn't do any better, or something more specific I've simply not run into?
Where "reverse engineered" should be translated as "telnetted to the bitkeeper server/port and typed 'help'"
Why doesn't the gene pool have a life guard?
You can only split a subdirectory out into its own project if it's not related. If changes ever cross the boundary, you want history. If you just want to point someone to a single place for a checkout, you want the versioning system to have some notion of that.
Meanwhile: Burning a DVD is possibly slower or faster depending on the situation. It's a pretty stupid option either way, though. In this day, any time you're in a situation where using physical media is a better option, you've got a broken protocol. If I don't /want/ 12GB of history for images I'm not using (even if someone else might), I should not have that data at all.
I really think it would be much better and make a lot of people happier if git would just allow a shallow checkout which asked the server for more if needed (and that server can do the same from its origin if it doesn't have it either). Give people the option of setting up servers to do history processing for people who only want a shallow copy for 99% of the work they're doing. And if I want to check out "all history up to version 2.0, but nothing earlier than that", that would likely satisfy absolutely everyone for every day-to-day work anyone ever actually does.
Pulling entire histories is _always_ asking for trouble down the line. Subversion has the fatal flaw of keeping track of everything forever, even if no one wants it anymore (google "svn obliterate" for discussions). git solves this problem by simply not caring about the ramifications, but if the primary repository deems something necessary, EVERYONE gets it. At least with svn the problem is server-side-only.
-- 'The' Lord and Master Bitman On High, Master Of All
You shouldn't put in the things you can't diff.
So all the binary data etc. that is required to build an application should be managed seperately? Our GUI code is generated by a third party tool which stores its information (e.g. fonts) as part of a binary database. This belongs with the source code because the code needs to be modified in step with it. Having the log message is very helpful as it would take hours to work out the changes made between two of these files, because diff isn't useful. The files are 15MB in size.
SVN may not be the best choice for binary data but at least it is possible to put binary data into it. I would rather endure SVN's slowness than have to manually manage binary files. I believe that revision control could be better supported by operating systems. Two copies of every file are managed by a SVN checkout - the base file and the working file. If the filesystem could store these together then the cost would be halved (if the working file referenced the base file until the working file needs to be changed). The SVN tools would then be able to work much faster because the need for file comparison would be less common.
Unless the revision control system's performance is dreadful I think that all files should be in revision control.
To my knowledge, almost no one besides committers used Perforce to check out the Perl 5 source code. The documentation suggested using an rsync mirror. That's what I did.
I don't see how. This has very little to do with Perl modules, nothing to do with the CPAN, and everything to do with how people who hack on Perl 5 itself get its source code. (Very few of those use Windows, and they seem confident that Git on Windows is sufficiently usable.)
how to invest, a novice's guide
You guys know that, In the UK at least, git is a moderate insult with the same meaning as bastard?
Linus likes to name his projects after himself. Hence, "Linux" and "git" - the British slang meaning of the term being the one implied. I like the Oxford definition; "an arrogant or contemptible person". "bastard" has another meaning, though its slang use approximates the same :-).
Heh, you obviously didn't find any of the actual code used for the pre-historic import, the hostile import, the Raw Perforce Importer or the scarier SQL queries used to manipulate the data. Your program is far easier to understand :-).