patchwerk.org ============= ''social engineering for human beings'' A GUI (e.g. github) is actually usually the correct way to do repository administration. There are a finite amount of things that humans should really do, and while they should be be a group of atomic actions put together to form a workflow policy, doing something like "splitting a patch" is easily doable via a GUI (even if you're an idiot) and hard to get right from the command line (even if you're a pro). meet patchwerk.org! ------------------- patchwerk.org is a site for writing code in a collaborative way. Use the advantage of the mercurial system to write code with other people! Lots of eyes is better than your two! Being the only engineer on a project lets you write code however you want to. If you're a good engineer (you know, the kind that writes tests and does all sorts of other checks to make sure you're software works), then being the only developer allows you to make applications really fast since the only bottleneck for decisions is you. But something is missing....breadth. If you write code, it is how you think the problem should be solved which depends on your definition of the problem. ''Even the best engineers have something to learn!'' Even if you're the best hacker there is, others might think of valuable features that you didn't. Some of these you might disagree with. Others you might think, ''Why didn't I think of that?'' This is why social engineering is useful. While of course having others catch silly mistakes is important, and particularly with programmers less experienced with a language, having interaction can help teach them the tricks of the trade, developing a deep understanding of the problem-space is an aspect of engineering that has no limit of mastery. About Distributed Version Control --------------------------------- Since its inception, distributed version control systems (DVCS), such as mercurial, git, and bzr, have had proponents stating that these had solved a vital problem in computer science. While this is not untrue, it is usually not the problem that proponents of these systems say it is. The problem that DVCS solves is the problem of replication on top of that of regular version control systems (VCS). All of the sudden, a repository need not be canonical. I can easily just copy the repository and apply whatever changes I want. Others can contribute to my repository and, in fact, the repository that I copied from is no more canonical (save by convention) than my repository. If version control is an infinte spool of rope to hang yourself with, then DVCS is a rope factory. The problem that DVCS *doesn't* solve is that of putting things back together. Sure, I can work on my clone and you can work on your clone and others can clone from us ad infinitum. But what if I want a change from you? DVCS proponents point out (correctly) that DVCS have much better tools for merging these changes than non-distributed VCS. However, ultimately merging is a problem that requires a human in front of a screen making decisions on whether and how disparate changes can work together. If you add a new file and I add a new file, then probably (but by no means guaranteed!) there will be no conceptual conflict between our changes. If you and I edit the same file but don't touch the same lines, the possibility of conflict is greater, but again is possibly unlikely (depending on what we did of course). If we edit an overlapping region of a file, then this just has to be sorted out. In special cases, it can be said that there is an easy merge algorithm... *...but even if we have changes in completely separate files this does not mean that our changes will play nicely together!* So DVCS made it easy to develop a patch or similar to someone else's code. But, save for the trivial case where there are no changes to the master, DVCS has not solved the problem of how to put these hacks together. The problem of merging is too easily and too often reduced to an algorithmic problem. Sure, better tools save time and make our life easier. We like better tools! But until machines start thinking for themselves (I'll give them ten years), programming is about manifesting intention and therefore *merging changes is merging intention!* What is needed isn't better merging tools. Those are pretty good already. What is needed is... Workflow! --------- patchwerk.org is designed to give you workflow that feels natural and that you really don't have to think about. We give you all the ability to get around this workflow and do horrible things, so if you don't agree with how we see the universe, then you're certainly welcome to work however you want. Hey, it's *your* software! But we hope you find what we've done pretty useful, at least within the context of social software engineering. So what are we talking about here? Forget forks. It used to be that fork had a rotten taste to it. Sure, the meaning of a fork in github is entirely different from "I'm going to fork ${project} because you refuse to include my feature" or even from vendor branch. But when you're hacking on some code, all you really care about is ... - the changes you are making ...and... - the ability to use the code with your changes patchwerk makes this happen with patch queues and easy tools to use them. If you're making your own project, then you're probably working on the trunk (unless it's a large project). But if you're hacking on someone else's project, all you really care about is keeping it in sync with the source and getting your changes upstream. And if you're anything like me, append "as soon as possible" to that last sentence. So your changes are hosted in a queue of patches. You can have as many patches to a project as you want. They may require being applied in order ... or not. That's up to you. The best way to use patch queues is to have a patch associated with a single issue, like "implement 3d rendering" or "profile performance of web server" -- something that's atomic to a human, and to put these issues in some sort of issue tracker. This gives you a tight correlation between intent and manifestation. ''So how do I get started?'' Start a project on patchwerk.org and start hacking! Get other hackers involved, it's fun! -or- Find a project on patchwerk.org that you want to hack on! Once you have an account, just find some code you want to work on. Click on a place you want to edit (or you might notice the "Make a patch" button which will help get a clone of the code and a working patch queue on patchwerk.org and your local machine) and patchwerk will automatically make a patch queue for you! (Or if you already have a patch queue for that repository, make a new patch in it or append to an existing one). Once you're done editing, browse to your patch. If you think it's ready, there is a "Ready to sail upstream" button that will let the original repository curator know that your patch is ready (in your opinion, anyway) to be put in his repository. S/he might put it in straight-away. Or s/he might have things s/he wants you to fix before putting it upstream. Or s/he might not want the patch (sorry!). But you can keep your patch and repository regardless of what upstream decision makers think. patchwerk.org still allows you to fork -- we just don't encourage it as a first option. The reason that patch queues provide nice workflow as you can provide some guarantee that they apply cleanly. No, this isn't a contradiction about everything said above, that merging is a human problem not an algorithmic problem. You still want to have someone looking at these things before sending them upstream. But given the state of a repository and a patch, you can see if it applies cleanly or not. If it does, then you at least *can* push it upstream (whether you *should* or not is your call) and be assured it will apply cleanly. Each patch has a icon indicating whether the patch is up to date with the canonical tree (or trees, for advanced users). If it is, you're good to go. You can optionally have notifications sent when the source repository changes (either the file(s) your patches are in, just when there are conflicts, or more safely, the entire repository). If a patch doesn't apply after a commit, the patch will be labeled as broken and you'll receive a notification unless you opt-out. Once the patch is applied, it is removed from your active queue and put on the backshelf. You'll still have access to it, but you really don't need it anymore. Features -------- While you can't write a tool to solve problems, you can write a tool to help you think about problems. patchwerk.org is such a tool. We've bundled several bits of technology to help *you* develop software: - patchwerk uses the excellent mercurial version control system - patchwerk enables the excellent patch queue extension to mercurial. This enables you to hack on others' code while still keeping the source clean - patchwerk features excellent workflow for collaboration. As a project manager, when someone alerts you to a patch they want you to include, you can review the patch easily and can see if it is applied to your most latest revision of the code (or more! We'll get back to this in a second). As a contributor, you can easily check out a project you're interested in and start hacking right away. Then, when you want to share your hack, you ping the project manager(s) to look at your patch! We can't make sure it will be included, no matter how useful it is, but we do provide the tools to make sharing patches easily. - code review! Collaboration is really about talking about code. Some of this is pretty high level, but you also need to talk about the nuts and bolts. patchwerk let's you have discussions about files, patches, commits, or particular sections or lines in either of those. - splitting patches! Often what was thought of as one issue is actually two (or more). patchwerk.org provides an easy TTW interface for splitting a patch into multiple pieces (or, much easier, flattening a set of patches). - applying patches to multiple trees: it's never fun, but often there is a need for sizable projects to have multiple branches or (far worse!) to have the same source living in multiple places. While currently the latter is out of scope, patchwerk.org allows you to mark your patch for multiple trees. This is a bit of an advanced feature as now, instead of just one target, you have a few. On the other hand, it's better not to have to copy+paste when you don't have to. This feature is documented more completely on http://patchwerk.org Future features: If people like patchwerk.org, there's a lot that can be done to make this not only a great platform for collaborative development, but a great community. - Issue tracking: Something is wrong with your software! -or- You want to add a feature! Note it in our issue tracker! Once you adopt an issue, a patch will be set up in your queue and all commits on this patch will be automatically noted on the issue. Likewise, creating a patch will automatically trigger an issue to be created in the tracker. Finalizing a patch (either having it accepted upstream, or noting that it is finalized to whatever state) will automatically close the issue. Parallel to patches, issues can be forked and combined. I could talk about issue tracking all day, so stay tuned for excited updates here. - Wiki and mailing list: Projects need places to talk about themselves. Wiki pages serve as a great collaborative space for everyone to develop canonical documentation. The wiki can consume documentation featured in the package (in many formats) -and- editing the wiki can push the documentation up to the project (in some formats). (The workflow of this depends on the general project policy. The default case is to treat wiki documentation the same as other source code, so the wiki edit creates a patch which must be approved before putting it in the source). Archived mailing lists give a place to have high level conversations. When a mailing list discussion crystalizes, we provide tools to mirror the takeaways onto the wiki, notifying folks of where this now lives. And as you might imagine, our wiki and mailing lists play nice with the issue tracker and code review tools! - Continuous Integration! (coming soon!) More on patch werkflow ---------------------- Q: What if I basically like someone's patch but there's something I don't like about it? A: Easy. Since patch queues are first-class repositories, you can make a patch queue for their patch queue! Make a change to their patch and, if they decide to accept the change, you can end up with a patch you both agree on. You can even set it up so that if they accept your change, it automatically gets finalized to your source repository! (Assuming there were no other changes, of course). Q: Neat! Can I have patch queues to patch queues to patch queues? A: You can, of course, but we don't provide any front-end or functionality supporting this. A project is a piece of code. A patch queue is a collection of pending ammendments to that code. A patch queue to this is critiques of patches. Levels beyond this don't really mean anything more than critiques of critiques. Maybe if we get large enough projects to need this, we'll open up this functionality, but sometimes sanity is better than a complete lack of restrictions. Q: Can I edit code through the web? A: Yep! If you edit your own code, (be warned!) it will automatically be committed and pushed. So make sure that's what you want to do! If you try to edit someone else's code or patch, then you will automatically generate (or add to) a patch queue for that repository. Working through the web is great if you have some little change or annotation to make. If you're doing something bigger, you might be more comfortable checking out the code and editing with your favorite emacs. Q: What if I want to host my code elsewhere? A: Assuming your code is in hg, git, or svn, you can host it anywhere so long as it is publically check-outable. We provide mechanisms such that you can give a way to automatically push upstream. If you don't want to use these mechanisms, then you'll have to do this manually. Sorry! There are some problems that can't be solved. Q: Is patchwerk.org a federated system? A: You bet it is! patchwerk.org is an openID provider and so should work with any other patchwerk websites. They like to talk to each other, so you can certainly mirror and aggregate work on any server running the software. If you want to set up your own patchwerk site, the code lives at http://patchwerk.org/patchwerk/patchwerk.org (Cute eh?) Read the README for instructions on how to get started. You can find other of our tools in http://patchwerk.org/patchwerk/ if you're interested. HAPPY HACKING!