[esp-r] New ESP-r Code development instructions: Part 1 - The basics
Ferguson, Alex
Alex.Ferguson at NRCan-RNCan.gc.ca
Tue Mar 18 19:18:07 GMT 2014
As you know, I began migrating ESP-r from Collab.net to git-hub last month. That transfer is largely complete now. I'm passing on an initial set of instructions about how to work with the repository. If you're new to svn or git, please refer to the many comprehensive tutorials on the web.
These instructions cover the layout of the new repository, and basic check-out/check-in transactions under git and svn. I haven't yet jumped into merging, but I understand that merge operations are much easier on github than in the old repository. More details will follow.
- Alex
1. How to join the ESP-r development community:
If you haven't already done so, head over to https://github.com/ and create an account. Send me your username, and I will add you to the ESP-r community.
2. How to find ESP-r on github:
ESP-r is located here: https://github.com/ESP-rCommunity
In the ESP-r community, you will find the following ESP-r projects:
* ESP-rSource: The functional source code, installation scripts and test cases for building ESP-r.
* ESP-rModels: Training, exemplars and validation models associated with ESP-r.
* ESP-rDatabases: Core databases and climate files associated with the ESP-r distribution
* ESP-rDoc: Documentation associated with the ESP-r project
* ESP-rExtras: Extra files associated with the ESP-r project. Includes climate, model, training materials
* CHREM: The Canadian Residential Energy Model
Three of these projects (ESP-rModels, ESP-rDatabases and ESP-rDoc) were formerly subdirectories within the src directory in the old repository. I moved them into separate modules because there is a lot of commit traffic associated with these files, which makes it harder for developers to stay in sync with the master branch. These projects are still needed to build ESP-r; if you clone the ESP-rSource repository in git, git will recognize these as submodules and will pull them in automatically. I've also created a script that automates this for svn users (details below).
3. How to create a branch, and how to delete it
In github, you can create and delete your own branches. To do so, click on the branch button and enter an appropriate name:
[cid:image001.jpg at 01CF42BD.428F1840]
I strongly encourage you to delete your branches regularly. Once you changes have been pushed to the master branch, delete your branch and start new development. This will make it easier to stay in sync.
4. How to recover a branch from the old repository
I've kept a complete backup from the old repository; I can reconstitute branches (including their history) on the new repository. Let me know if you want me to resurrect a particular branch.
5. How to work with ESP-r source code USING SVN
Obtaining the code
github provides an SVN api that allows you to work with the ESP-r repository using svn commands. To download a local sandbox, use the checkout (co) command:
svn co https://github.com/ESP-rCommunity/ESP-rSource.git/branches/[My-Branch-Name<https://github.com/ESP-rCommunity/ESP-rSource.git/branches/%5bMy-Branch-Name>]
For example, to obtain a sandbox from the master branch (formerly development_branch), run:
svn co https://github.com/ESP-rCommunity/ESP-rSource.git/branches/master
To reduce the amount of merges that developers are responsible for, I've moved the training, validation, databases and climate directories into separate git modules. Git can automatically resolve these submodules, but with svn, you'll need to check them out manually into your sandbox. I've created a script called svn-fetch-modules.sh. To use it, move to your sandbox and run './svn-fetch-modules.sh'
ip-10-97-134-174:~ # cd master
ip-10-97-134-174:~/master # ls
.gitmodules .svn GPL.txt Install Readme Version src svn-fetch-modules.sh tester
ip-10-97-134-174:~/master # ./svn-fetch-modules.sh
svn-fetch-modules.pl:
path: ~/master
Checking out documentation via svn...-> doc..
A doc/README.md
A doc/archive
<SNIP>
A data/databases/readme_plant_dbs
Checked out revision 1.
Setting svn ignore properties for databases, doc, models...
property 'svn:ignore' set on '.'
ip-10-97-134-174:~/master # ls
.gitmodules .svn GPL.txt Install Readme Version data doc models src svn-fetch-modules.sh tester
Compiling
You're now ready to compile ESP-r. From the top directory in your sandbox (e.g. master), run ./Install.
NOTE: Under the new directory structure, run Install from the top-most directory (i.e. 'master') and not the src sub-directory (i.e. master/src).
Committing your changes
>From within your sandbox, run svn commit as you would normally. Note that if you're making changes to the data, doc, or models directories, you'll have to make separate commits from within these directories. This is necessary because these directories are now separate git modules.
Updating with changes on your branch
>From within your sandbox, run svn update as you would normally.
Merging changes
svn merge will not work on github. Instead, I believe you can use the web tools on https://github.com to pull changes from the master branch and to push changes back to your branch. However, I'm still working out the details. More info to come.
6. How to work with ESP-r source code USING GIT
Git is different than svn. It's more distributed, more robust and more powerful. But it's also more complicated and more confusing for beginners. I recommend reading:
* Why svn users should switch to git: http://heartmindcode.com/2011/03/27/svn-users-why-you-should-switch-to-git/
* Git for svn users: http://git.or.cz/course/svn.html
Update git
Some older versions of git have proven unstable. Github recommends using git version 1.7.10 or later. Check your version using the command 'git version':
ip-10-97-134-174:~/ESP-rSource/src # git version
git version 1.7.12.4
Please update to a more recent version if your copy is out of date.
Obtaining the code
The git command 'clone' is similar to an 'svn checkout'. To clone the master branch, use the command:
git clone --recursive https://github.com/ESP-rCommunity/ESP-rSource.git
EXAMPLE:
ip-10-97-134-174:~ # git clone --recursive https://github.com/ESP-rCommunity/ESP-rSource.git
Cloning into 'ESP-rSource'...
<SNIP>
Receiving objects: 100% (45051/45051), 63.79 MiB | 30.61 MiB/s, done.
Resolving deltas: 100% (33799/33799), done.
The git clone command may seem like an svn checkout, but it's more comprehensive. When you run 'git clone', you actually make a fully-functional copy of the entire repository (including all branches). Your local repository is now a complete backup of the entire history on all branches of the repository. While your cloned repository contains a link to the github repository that allows you to synchronize changes (see git pull and git push, below), it also functions as a stand-alone repository.
Inside the ESP-rSource Repository, you'll find the master branch (formerly development_branch) source code:
ip-10-97-134-174:~ # cd ESP-rSource
ip-10-97-134-174:~/ESP-rSource # ls
.git .gitmodules GPL.txt Install Readme Version data doc models src svn-fetch-modules.sh tester
Git will only show you one branch at a time in this repository. If you've cloned with the command above, you will find git has created a copy of the master branch in you the ESP-rSource directory. To find out which branch you're on, use the 'git branch' command
ip-10-97-134-174:~/ESP-rSource # git branch
* master
You can also see a list of all branches in the repository:
ip-10-97-134-174:~/ESP-rSource # git branch -a
* master
remotes/origin/Achim_Geissler
remotes/origin/Alex_Ferguson
remotes/origin/HEAD -> origin/master
remotes/origin/NRCanOpt
remotes/origin/h3k_switch
remotes/origin/master
To switch to another branch, use the command 'git checkout'
ip-10-97-134-174:~/ESP-rSource # git checkout Alex_Ferguson
Branch Alex_Ferguson set up to track remote branch Alex_Ferguson from origin.
Switched to a new branch 'Alex_Ferguson'
ip-10-97-134-174:~/ESP-rSource # git branch
* Alex_Ferguson
master
For svn users, this is one of the most confusing aspects of git. A 'git checkout' doesn't download any code-it just switches between branches on a repository you've already cloned.
Note: you can also clone the repository and checkout the branch in a single command:
git clone -b [My-Branch-Name] --recursive https://github.com/ESP-rCommunity/ESP-rSource.git
This is equivalent to running 'git clone' and then 'git checkout [my branch name] '
Compiling
You're now ready to compile ESP-r. From the top directory in your branch (e.g. ESP-rSource), run ./Install.
NOTE: Under the new directory structure, run Install from the top-most directory (i.e. 'master') and not the src subdirectory (i.e. master/src).
Committing your changes
The git 'commit' command is analogous to an svn commit, except it only commits it to your local repository. Remember, 'git clone' creates a fully-functional repository on your local machine. To pass your changes up to git hub you must also issue a 'git push'command.
ip-10-97-134-174:~/ESP-rSource/src # git commit Readme
[Alex_Ferguson 3876570] Test commit
Committer: fergusoa <root at ip-10-97-134-174.ec2.internal>
Your name and email address were configured automatically based
on your username and hostname. Please check that they are accurate.
You can suppress this message by setting them explicitly:
git config --global user.name "Your Name"
git config --global user.email you at example.com
After doing this, you may fix the identity used for this commit with:
git commit --amend --reset-author
1 file changed, 1 insertion(+)
ip-10-97-134-174:~/ESP-rSource/src # git push
Username for 'https://github.com': fergusoa
Password for 'https://fergusoa@github.com':
Counting objects: 7, done.
Delta compression using up to 16 threads.
Compressing objects: 100% (4/4), done.
Writing objects: 100% (4/4), 369 bytes, done.
Total 4 (delta 2), reused 0 (delta 0)
To https://github.com/ESP-rCommunity/ESP-rSource.git
e84789b..3876570 Alex_Ferguson -> Alex_Ferguson
Remember: Any changes you commit to your local git repository are not available on github until you issue a 'git push' command!
Note that if you're making changes to the data, doc, or models directories, you'll have to make separate commits from within these directories. This is necessary because these directories are now separate git modules
Updating with changes on your branch
You can refresh your local repository with changes on github via the git pull command. Note that you may see traffic on other branches --- this is normal because your local git repository is a clone of the entire github repository (not just your branch).
ip-10-97-134-174:~/ESP-rSource/src # git pull
remote: Counting objects: 2, done.
remote: Compressing objects: 100% (2/2), done.
remote: Total 2 (delta 0), reused 0 (delta 0)
Unpacking objects: 100% (2/2), done.
>From https://github.com/ESP-rCommunity/ESP-rSource
8e205f1..0eb7055 Achim_Geissler -> origin/Achim_Geissler
Already up-to-date.
Merging changes
You can use the web tools on https://github.com to pull changes from the master branch and to push changes back to your branch, or using the git merge command. However, I'm still working out the details. More info to come.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.strath.ac.uk/archives/esp-r/attachments/20140318/715c3097/attachment-0001.html
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.jpg
Type: image/jpeg
Size: 21268 bytes
Desc: image001.jpg
Url : http://lists.strath.ac.uk/archives/esp-r/attachments/20140318/715c3097/attachment-0001.jpg
More information about the esp-r
mailing list