Slashdot Mirror

← Back to Stories (view on slashdot.org)

Ask Slashdot: How Best To Synchronize Projects Between Shared Drive and PCs?

Posted by timothy on from the phrase-your-answer-in-the-form-of-a-cookie dept.

Koookiemonster writes "Our company has many projects, each one with a folder on a Samba drive (Z:\). Our problem is syncing only the programmers' current projects (~30 at any time) between Z:\ and their C:\Projects\-folder on five Windows 7 laptops. If we sync the whole Z:\-drive, our projects folders would be filled with too many subfolders, making it difficult to navigate. The folders contain OpenPCS projects (PLC) and related files (Word, Excel, PDF documents); a common project folder is 50 MB. Is there any easy to use, low-budget sync software with scripting, so that we could e.g. only sync folders that exist locally?" (Read more details, below, of what Koookiemonster is looking for.) "Many programs do support selective sync, but choosing what to sync is awkward; projects and who works on them change daily. It is important that subscribing to a project is as easy as copying it from Z:\ to C:\projects\. The Z:\-folder with all of our current and past projects is located on a desktop PC running Ubuntu Linux. It can share files e.g. via Samba or FTP. All PCs are on the same (W)LAN. Off-site backups of Z:\ are taken care of via rsync. The company has three programmers, who usually handle their own projects alone, but very often others need to add files to projects. Bigger projects need more programmers. Currently we use FreeFileSync with a custom piece of Javascript to make batch files that synchronize e.g. folders C:\projects\123_ProjectName\ and Z:\123_ProjectName\ if the local folder exists. However, that solution lacks versioning, real-time sync and deletion support. It only syncs when we press a button, and then older files are overwritten by newer files (two way sync; older files go to a "sync-deletions"-folder).

PS. Bonus points for solutions that allow renaming project folders without renaming them on all laptops."

36 of 238 comments (clear)

  1. BTsync? by pete.zhut · · Score: 2

    I don't know if this will do everything you need, but I have been using it at home to backup three machines to my NAS. Seems to work quite well.

  2. stop trying, use git instead by Anonymous Coward · · Score: 5, Insightful

    you are welcome

    1. Re:stop trying, use git instead by camperdave · · Score: 5, Informative

      stop trying, use git instead

      ... or one of the many alternatives.

      --
      When our name is on the back of your car, we're behind you all the way!
    2. Re:stop trying, use git instead by GameboyRMH · · Score: 2, Insightful

      Correct answer almost on the first post, nice work ("svn" would also have been acceptable.)

      TFA is describing a horrible clusterfuck that must be replaced with proper revision control. You don't build on top of this, you KILL IT WITH FIRE!

      --
      "When information is power, privacy is freedom" - Jah-Wren Ryel
    3. Re:stop trying, use git instead by luis_a_espinal · · Score: 2

      You'll have to excuse him, he comes from Windows Land :)

      Meh, I know you are posting in jest, but honestly, I've seen people from Unix-land also having no clue about source control/revision control systems. Proficient software people (Unix, Windows or whatever) know how and when to use such a system. Everybody else, well, you get what the submitter is describing :/

    4. Re:stop trying, use git instead by Darinbob · · Score: 2

      Or just about any source code control system (most allow binaries as well). I can't think of any that don't allow syncing just files that are wanted. Further, these systems are designed to safely share the files, not just "sync"; files can be locked and made exclusive, or conflicts between multiple authors resolved, etc. This is essentially technology that's 40 years old.

    5. Re:stop trying, use git instead by Darinbob · · Score: 3, Insightful

      That's truel However I definitely have seen the syndrome of people not acknowledging software not created by Microsoft, or asking for bizarre or counterproductive requirements (sharepoint). Most Unix users who don't understand this stuff at least tend to follow along with whatever the team uses or project manager suggests.

    6. Re:stop trying, use git instead by Bacon+Bits · · Score: 2

      I wouldn't use git. Git requires you to always clone the entire repository. That's fine if the repository is just source code and text files, but the more binary files you have the less attractive this can be.

      I still favor svn over git myself, but it just suits my workflow better than git.

      --
      The road to tyranny has always been paved with claims of necessity.
    7. Re:stop trying, use git instead by gnapster · · Score: 4, Informative

      svn is appealing in this case because you can check out subdirectories of a repo without downloading the whole thing, which is more similar to his current organization.

    8. Re:stop trying, use git instead by alexhs · · Score: 2

      syndrome of people not acknowledging software not created by Microsoft

      Why would anyone install some perfectly working third-party application on a pristine Microsoft system ?
      I suggest Microsoft Windows Briefcase.

      --
      I have discovered a truly marvelous proof of killer sig, which this margin is too narrow to contain.
    9. Re:stop trying, use git instead by Gr8Apes · · Score: 4, Informative

      You apparently don't have the slightest clue how git works, nor how to use it properly. I certainly am no git guru, but have no issues with using multiple git repos in projects, building sub-systems out of it, nor branching and merging, which it truly excels at. I wouldn't wish SVN on anyone that wants to use branching. It's about as brain-dead as it can get and still "function", barely. The only thing SVN has going for it is that it is almost atomic, whereas CVS, MKS, etc, absolutely aren't. ClearCase is a separate system that can be used, but requires a full time expert admin for anything more than basic code repos. At least it handles branching almost sensibly. Mercurial was in the running, but at the time git had (and still has) a larger user base with active improvements in various tools associated with it for my needs, thus I chose git.

      Learn a tool before you wrongly despise it publicly. You are wrong on every count:

      • You don't get a full copy of everything on every system if you don't want it. You can just check out a single branch.
      • Every copy is not a full backup. See previous line item.
      • Selective work is easier in git, you can clone the item(s) desired, branch locally, and merge when complete, pushing only if desired.

      Lastly, you can essentially "checkin" every change for a full history of what you did on a local branch, revert, merge other branches, etc, with no effect on the main dev branch(es). And you can do all this without even being connected to a "main" server. Maintaining parallel branches with constant merges is cake compared to SVN and other central repo schemes. There really is no situation where I'd rather use SVN or anything like it that I can think of, when I have a choice.

      --
      The cesspool just got a check and balance.
    10. Re:stop trying, use git instead by philip.paradis · · Score: 3, Informative

      Perhaps you haven't spent enough time with git. I've used it for years to manage data stores with content ranging from rST documentation bound for rendering to a very widely read open source "howto" site (with constant edits and merges from a small team of technical writers) to large scale development projects. In fact, I use it for all my file and source control to this day, and my employer's dev group uses it as well to manage a rather extensive codebase responsible for driving an enterprise cloud hosting provider's operations.

      If you've had problems with git, you should be aware that there's a huge community out there ready and willing to assist you with whatever workflow you've decided to adopt. One of the nicer things about git is the fact that you can use it in a very SVN-like manner if you like, or you can make as many branches as you want and manage things in a very distributed manner. Again, I suspect your primary problem is lack of experience.

      --
      Write failed: Broken pipe
  3. OwnCloud is pretty good by charnov · · Score: 2

    https://owncloud.org/

    Keep the masters in a private cloud and sync to it from your PCs. Git and other multi-user SVN is an idea, too. Also, SharePoint is excellent (but lots of overhead).

    --
    [RIAA] says its concern is artists. That's true, in just the sense that a cattle rancher is concerned about its cattle.
    1. Re:OwnCloud is pretty good by Amouth · · Score: 2

      I'd like to chime in on ownCloud. It works but it also doesn't

      Owncloud fails in a large multi user environment over low speed connections, and with office files if the users are working in the same folder that syncs. We tried it, and for single in a folder it worked great, once we added others (especially with low speed or low quality connections, aka traveling users) it failed miserably. We had many many many conflicts for no reason. many many 0 byte files do to cross sync issues.

      We ended up canning it because managing it and cleanup became too much of an issue. We now use DropBox (i know not open source and you pay) but the reality is it just works. And it has the selective sync they are looking for.

      --
      '...if only "Jumping to a Conclusion" was an event in the Olympics.'
    2. Re:OwnCloud is pretty good by butonic · · Score: 2

      Again, try the new version 1.4. As discussed in https://github.com/owncloud/mirall/issues/29 inotify on linux does not guarantee that the sync client will get notified of file changes under all circumstances, so we do need to poll. Any constructive hints are welcome.

  4. Revision Control? by Anonymous Coward · · Score: 5, Funny

    This sounds like the ramblings of someone who has never heard of revision control.

    1. Re:Revision Control? by camperdave · · Score: 4, Insightful

      Maybe he's one of the Ten Thousand

      --
      When our name is on the back of your car, we're behind you all the way!
  5. A Small Voice In the Wilderness Calls Out by the+eric+conspiracy · · Score: 4, Informative

    USE VERSION CONTROL!!!!

    Git
    Mercurial
    Perforce
    Subversion
    Vesta
    CVS
    ClearCase
    VSS
    StarTeam

    The choices are legion. What you are doing is not a choice.

    Pick a version control system and your life will be much easier (after the learning curve).

    1. Re:A Small Voice In the Wilderness Calls Out by Anonymous Coward · · Score: 3, Funny

      Why close? We can now start the entertaining flamewars about which version control software is the only reasonable one! ;-)

    2. Re:A Small Voice In the Wilderness Calls Out by the+eric+conspiracy · · Score: 4, Informative

      I've used many of the above. Here are some comments (the first four are free):

      1. Git - really capable tool, with opaque command model. Maybe not for newbs. Managers hate using it (one reason to like it). Has good cloud support with GitHub.

      2. Mercurial - pretty easy to use. Great if your projects don't have large binary files. It's a good introductory decentralized repo tool.

      3. Subversion - pretty good if you want a centralized repo.

      4. CVS - your father's version control system. Still way better than not having version control.

      5. ClearCase - good in the hands of a ClearCase specialist. Would not be my choice for a small team of version control newbs.

      6. VSS - My experiences with it were not good. I hear it's improved. I'd look at it ONLY if I was in a Microsoft only shop.

  6. Subsetting a repository of files by Empiric · · Score: 4, Informative

    Sounds exactly like what an RCS like Subversion is good for.

    Each user pulls down the directories relevant to him/her from the overall repository, updates at will from the central source, and pushes up changes at will with a couple of mouse clicks.

    TortoiseSVN will even give you handy little icons on your local folders in Explorer to tell you if what you have in your local directory isn't synced with the central server, and it's two clicks to make that happen. I actually think you don't want to "force synchronization" on an ongoing basis, seems like a great way to overwrite a lot of your developers' (and others') ongoing work.

    --
    ~ Whence do you come, slayer of men, or where are you going, conqueror of space?
    1. Re:Subsetting a repository of files by Empiric · · Score: 2

      Hey, I get enough attempts at that here just from being theist. ;)

      --
      ~ Whence do you come, slayer of men, or where are you going, conqueror of space?
  7. Why not use source control? by gmezero · · Score: 2

    We used to use ViceVersa Pro to sync our team but eventually moved over to Plastic SCM which has been friggin' awesome. It not only supports code, but also art assets. Plus it has the best support for branching. One team can be working on a branch specific to one project, while another works on a second branch while the main trunk stays clean and build-able. You can even have developers run their own local repository on their desktop/laptop and have them replicate/merge either on a schedule or when they connect to the LAN (if they work offsite alot).

    You only bring stuff back to the main trunk when you're ready to merge a branch back in. You can even merge branches separate from the trunk. We check everything into it, code, art, and our Doxygen output. It's been a time save on orders of magnitude.

    Not trying to be a sales pitch, but you should check it out.

  8. Don't try syncing files by MobyDisk · · Score: 4, Informative

    Syncing files like this is a mess. Perhaps you should look beyond share drives. You are trying to solve the technical problem, but if you step back you might see a business problem. Consider 3 alternative approaches:
    1) Keep the files on the share drive and do not mirror them locally.
    2) Use a source control system (Ex: GIT, TFS, Subversion)
    3) Use a groupware / content management system / document management system. ( Ex: Sharepoint, Confluence, QDMS, Lotus Notes, Microsoft Exchange, Drupal, SAP, Groupwise)

    Knowing the right terms helps find the software you need. Here are some links to Wikipedia which has the right terms, and some lists of software:
    http://en.wikipedia.org/wiki/Content_Management_Systems
    http://en.wikipedia.org/wiki/Document_management
    http://en.wikipedia.org/wiki/List_of_collaborative_software#Comparison_of_notable_software

  9. Re:Wow, Windows is really still that bad? by mrbene · · Score: 2

    Robocopy ( http://technet.microsoft.com/en-us/library/cc733145.aspx ) is included in all desktop versions of Windows (so, not RT or Phone). Extensive copying/moving/mirroring options, CLI-only. Great for integrating into scheduled scripts.

    I'd still agree with other poster here, and recommend git over Robocopy to the OP. However, Windows does have a robust tool for syncing files between multiple computers built in.

    In fact, Robocopy has been available since Windows NT 4.0.

  10. It's sad by msobkow · · Score: 5, Insightful

    It's sad to see an entire team of supposedly "professional" developers which have never heard of version control.

    You can't even blame it on the Windows environment -- MSVS supports hooks for several version control systems either natively or through plugins/addons.

    This whole story just reeks of some manager saying "We can't afford to set that up -- it would take too much time" any time someone has suggested it.

    Because I flat out refuse to believe the entire team doesn't know any better.

    --
    I do not fail; I succeed at finding out what does not work.
    1. Re:It's sad by Freshly+Exhumed · · Score: 2

      As soon as I saw all those drive letters in the guy's summary I just knew where this was going... very sad indeed when coders don't have a clue about the tools that obviously suit them best.

      --
      I deny that I have not avoided attaining the opposite of that which I do not want.
    2. Re:It's sad by WuphonsReach · · Score: 4, Interesting

      Because I flat out refuse to believe the entire team doesn't know any better.

      It's way more common then you'd think. I worked for a company back in 2000, with close to 20 developers. Not a single one of them knew how to do version control. Instead, they all resorted to making copies of files, tacking on dates or times or just numbers, and relying on the backup tapes if they had to undo a particular change. It was hellish and you'd hear at least one conversation per week where they were trying to figure out who had the latest version of XYZ.

      I rolled out VSS (hey, it was 2000, our choices were limited) with SourceOffSite for the remote workers. If I did it today, it would be either Hg, git or svn.

      Very few schools back in the 90s or early part of the decade taught VCS concepts or forced students to use them. This is slowly changing with the advent of things like 'github' which has a big mindshare and introduces people to the concept of VCS.

      --
      Wolde you bothe eate your cake, and have your cake?
  11. SFFS by zzsmirkzz · · Score: 2

    If you don't want to do Version Control as others have suggested then I recommend Super Flexible File Synchronizer. It is a great product with lots of options in regards to what does and does not get sync'd. It is inexpensive to boot. http://www.superflexible.com/ftp.htm

  12. If you can't use version control... by neminem · · Score: 2

    I suggest finding the person who says you can't use version control, and locking him in a room until he either changes his mind or ceases to be a problem. Then, use version control.

  13. What are your programmers doing? by damn_registrars · · Score: 3, Funny

    You said you have programmers working on these projects. They probably each have their own preferred way to do this, why not ask them? If they can't come to a consensus, you could have them write their own solution.

    --
    Damn_registrars has no butt-hole. Damn_registrars has no use for a butt-hole.
  14. An analogy. by pclminion · · Score: 5, Funny

    "I'm a contractor. I have a team of carpenters who are tasked with building a house. It seems this is going to require the driving of a large number of nails. My team of carpenters would like to know what sort of tool or mechanism would work best to drive these nails. Right now, we have one guy who holds the nail while another guy hits it with his thermos. This does eventually drive in the nail, but 90% of the time the nail bends, and it's denting our thermoses. I wonder if there exists some genius, super-carpenter bad-ass out there who might be able to suggest a better way."

  15. git-annex by Horus107 · · Score: 2

    I second git. git scales badly with large files. If this is a problem, you could have a look at git-annex http://git-annex.branchable.com/ The concept needs some time to grasp, but it's really powerfull.

  16. DO NOT USE GIT! Or you will be sorry. by Kaz+Kylheku · · Score: 2

    Git is absolutely not a good first version control system for people who are clueless about version control. (Such as, evidently, your developers).

    Git requires prior experience with at least two simpler version control systems. In git, you often run into scenarios that require you to understand its complicated repository representation so that you can choose the best steps to unravel them, based on understanding the ramifications of each approach.

    The implementation of git is not hidden from the user behind a robust set of "no brainer" use cases.

    The decentralized model alone will confuse the heck out of workers with no prior version control experience.

    Use a system that has a centralized server from which working copies are checked out, like Subversion.

  17. Maybe something like git-annex is right for you? by krakrjak · · Score: 2

    You might want to checkout git-annex: http://git-annex.branchable.com/

    It handles the idea of larger repositories with disconnected parts. You get git versioning of files and the ability to replicate portions of the data at will.

  18. Unison? by hobarrera · · Score: 2

    I agree that a VCS is the best choice, as suggested above. If, for some reason, you don't want to use that (eg: too many binary files grow you repo too much), unison is a great choice. It uses rsync to sync files, and keeps track of which files where modified on which side.