[esp-r] Changes coming to ESP-r central

Thu Feb 6 21:09:54 GMT 2014

Good afternoon from snowy Ottawa!

For eight years now, this community has been collaborating on ESP-r code development through a shared code repository (that we refer to as ESP-r central).  My organization, Natural Resources Canada, has paid to host that repository on third-party servers.  The arrangement has worked really well, and the community has contributed nearly 10,000 revisions to the ESP-r code base.

However, changes in my department's procurement rules mean that Natural Resources Canada can no longer continue to pay for this service. After considerable discussion, we have agreed to migrate the ESP-r repository to another, free service for open source software. You will still be able to access and contribute to ESP-r source code online, and the data in your branches will be preserved. But there will be some changes in how you interact with the code, and how you submit it for inclusion in development_branch.

Here are the top 10 things you need to know right now:

1.      What's changing?
We're moving ESP-r source code from a paid service provider (Collab.net, also known as Cloudforge, Codesion, and cvsdude) to GitHub (https://github.com/), a provider that offers free services to open source projects. ESP-r will remain open source. It will still be accessible on the web. And you can still use version control tools (such as svn) to download it, modify it, and improve it.

2.      When will this change happen?
Our collab.net subscription expires on February 22nd. We will move the repository before then.

3.      Why do we need to move?
Natural Resources Canada has paid for these services since 2006. But new rules in the Canadian Government prevent us from paying for third party services when comparable in-house capabilities are available. But while we do have free svn servers for our use, they also sit behind government firewalls and are not easily accessed by third parties (that is, you).

4.      Why GitHub?
There are a lot of version-control providers today. But there are also good reasons why we've chosen GitHub:

*         Github runs on git, but they've also built a full-featured subversion API. This means that if you don't like git, you can continue to use svn to manage your code.

*         Github is self-serve, which means that you can create and manage your own account. It allows us to create an 'organization' for the entire ESP-r community, and organize those communities into teams that help manage parts of the ESP-r source code. We can also appoint multiple administrators to the organization, to ensure the community can continue to function even if a principle supporter withdraws.

*         Github has a lot of investors, and it's making money. That matters to us because it means GitHub is more likely to be around for a while, and continue to offer services to communities like ours.

*         Github makes money from private companies who want to keep their source code private. But GitHub is built on open-source software, and their business model includes a commitment to supporting open-source projects.

*         Many other well-known projects are hosted on git, including OpenStudio (https://github.com/NREL/).

5.      What will the new repository look like?
The new repository will be hosted under an organization called 'ESP-rCommunity' (see: https://github.com/ESP-rCommunity/, this currently contains some test repositories that we're using to figure out how to migrate ESP-r to github) . Within there, you will find a source code repository, along with directions to download the code via 'svn checkout' or 'git clone'. There's also some other features, including a wiki, a discussion space and some analytics. You'll be able to create your own branches, and github will help you figure out how your branch differs from trunk, and how you can synchronize changes and submit contributions.

When you compare the new structure to the one we used on collab.net, you will see some changes:

*         We'll probably make some changes to the ESP-r directory structure to cut down the traffic in the src directory and make it easier for developers to keep up to date. Expect to see the Install script, training, validation, manual, archive and doc directories moved alongside the src and tester directories.

*         We're going to preserve the revision history on development_branch/src, and we can create snapshots that represent the current states of the development sub-branches in use today. This means that the new repository will not contain the history associated with your sub-branch.

*         You're going to be able to create and delete your own branches. This will make it easier to start work with the latest code on development_branch. After your development is done and you've submitted changes, you can chose to delete your branch and start fresh from a new copy of development branch.

*         You'll also see an end to email commit notifications. For better or worse, GitHub has decided that this feature is not on their roadmap. To find out what's happening in ESP-r source code, you'll have to visit GitHub.com. Once there, you'll find a useful dashboard showing development activity.

6.      Are we going to lose anything?
No. We've backed up the complete history of the repository back to 2006, including revisions and log messages on all branches. That file is too big to post anywhere. But I can share it with you, or I can query your branch for specific revisions or log messages if needed.

7.      Will this move orphan outstanding modifications on your branch?

No. If you have outstanding changes in your sandbox or on your branch, you have the following options

a)      Best option: Submit your changes to Ian Beausoleil-Morrison as per the existing procedure. If he receives them before Feb 14, he will fold them into development branch. This will give you a seamless transition to the new repository.

b)      Next-best option: Not ready to submit? Commit your changes to your existing branch. We'll create a branch for you in the new repository, and I will patch it for you to make it a snapshot of your existing branch. Your development can continue.

c)      Alternatively: Create your own branch as a copy of the current state of development branch. Migrate your changes in manually.

d)     If all else fails: Abandon your development and start over. Sometimes everybody needs a fresh start.

8.      What do you need to do to during this migration?

                                i.            Commit your changes: If you have outstanding revisions to submit, choose one of the options under item #7. If you choose option b) - snapshot of your current branch, be sure to let me know.

                              ii.            Create a github account: Create an account at GitHub (https://github.com/), and send me an email with your username. I'll add you to the ESP-r community.

                            iii.            Choose SVN or Git: Decide if you want to use Git or SVN to manage your source code. Git is more flexible and powerful. But it's also less intuitive. There are many online resources to help you decide (e.g: https://git.wiki.kernel.org/index.php/GitSvnComparison); if git seems like too big a mountain to climb, you can continue to work with svn. You can also switch from one to another at any time.

                            iv.            Use new checkout procedures: This migration will require some changes to how you check-out the repository and submit your changes. Details will come next week.

9.      What if you think this is a terrible idea?

I know our transition onto versioning control at cvsdude was rocky for some developers, and it's possible that you may find these further changes unnecessary and unwarranted. We want to hear your concerns. The archivist (Ian Beausoleil-Morrison) will entertain other ways that groups may use to obtain and contribute code to ESP-r. Contact him at ian_beausoleil-morrison at sbes.ca<mailto:ian_beausoleil-morrison at sbes.ca> to find out more.

10.  Will this change be smooth and painless?
I'm trying make this transition as easy as possible. But I'm sure there will be a bump or two along the way. I ask for your patience and understanding.

I'm sure many of you will have questions; please send them to me and/or Ian Beausoleil-Morrison.

- Alex

Alex Ferguson
Housing Team, CanmetENERGY
Natural Resources Canada
+1 613 995 3294
alex.ferguson at nrcan.gc.ca<mailto:alex.ferguson at nrcan.gc.ca>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.strath.ac.uk/archives/esp-r/attachments/20140206/93046aeb/attachment-0001.html 