Month: July 2012

GIT examples

git

I have been working on gitlab recently and needed to write down some git examples, so here is what I came up with:

1. To roll back files from your last commit

If the commit ID is ce4c1645cad94cc69221607201bd86c8f33b6cc0, run the following command

git reset --hard ce4c1645cad94cc69221607201bd86c8f33b6cc0

note: git reset without the –hard option resets the commit history but not the files. With the –hard option also files in working tree are reset

To then push these files back to the master branch

git commit --all
git push origin master --force

2. To recovery one file from a previous commit

To find out what files have been deleted from the previous commit, run

git log --diff-filter=D --summary

This should give you information about which commit the file was delete from. Once the commit ID is found, run the following command:

git checkout ce4c1645cad94cc69221607201bd86c8f33b6cc0 -- version_one_file_2

where ce4c1645cad94cc69221607201bd86c8f33b6cc0 is the commit ID and — version_one_file_2 is the file.

Then run the following commands:

git commit
git push

3 . Adding a FILE to a git repo

This is a very simple example but an important concept. Commands can vary depending on this example on what you are trying to do

create a new file

touch this_is_a_new_file

add the file to the git repo

git add this_is_a_new_file

this adds the file to the current working git directory

git commit this_is_a_new_file

this pushes the file to the master branchh

git push origin master

4. I deleted a file 3 commits ago. How do I recover that file?

How to reproduce the example

    1. clone git repo
      see above :)
    2. create a new file A
      touch A
    3. Add the file to the git repo
      git add A
    4. Commit and push the file A
      git commit A
      git push origin master
    5. create a new file B and delete A
      touch B
      git add B
      git rm A
    6. Commit and push the files
      git commit --all
      git push origin master
    7. create a new file C
      touch C
      git add C
    8. Commit and push the file C
      git commit --all
      git push origin master

Now try to find file A by running this command

git log --diff-filter=D --summary

it will show you what files have been deleted.

You need to checkout the previous branch; i.e. the commit before the deletion
You then need to make sure the file you are recoverying is the one you want
You then need to switch back to the master branch
You then need to add the file to the master branch
You then need to commit the recovered file
You then need to push everything to the repo

so..

git log --diff-filter=D --summary 
git checkout cdeb3d757f3adcc346da2ab171a825c113bdb50b~1 A

# note the ~1 rolls back to the previous commit. ~2 would go back 2 commits etc..

this just grabs that file in that commit and does not change branches

git branch

# check what branch you are on

git add A

# to add A back to the master branch

git commit A
git push origin master

5. Create a branch, adding file that does not conflict and adding the files to the master branch

Run git branch to show which branch you are using

git branch
* master

This shows you are using the master branch. To add a branch, run the following command

git branch new_branch
git branch

> * master 
  new_branch

The new_branch branch has been created. You have not checkout the new branch yet. First, the following command shows the files in the master branch.

git ls-files

 >INSTALL
  README
  config.xml

To switch to the new branch and to add a new file, run the following commands

git checkout new_branch

>Switched to branch 'new_branch'

git branch

> master
  * new_branch

notice the * is now next to the new_branch. This is to show which branch you are working on.

You can check the files are the same as the master branch by running

git ls-files

>  INSTALL
   README
   config.xml

Add a new file to the new_branch

touch file_added_to_branch
git add file_added_to_branch
git commit file_added_to_branch
git push origin new_branch

run the following to list the files in the new_branch

git ls-files

>  INSTALL
   README
   config.xml
   file_added_to_branch

switch to the master branch and list the git repo

git ls-files

>  INSTALL
   README
   config.xml

Notice that the file file_added_to_branch is not in the master repo. To add this file, you can merge the new_branch to the master repo by running the following command

 git merge new_branch

Updating 76a5cab..30c41cc Fast-forward 0 files changed, 0 insertions(+), 0 deletions(-)

create mode 100644 new_branch_file

Note: You have to be on the master branch to merge to the master. We changed to the master branch before running the above git merge command

push the files to the master branch on the git server

git push origin master

The following shows the branch files and how to delete any branches

git ls-files

>  INSTALL
   README
   config.xml
   new_branch_file

git branch

> * master
  new_branch

git branch -d new_branch

Deleted branch new_branch (was 30c41cc). 

git branch 

>  * master

Deleted the branch on the gitlab server

git push origin :new_branch
To git@puppet-gitlab:oliver.leach/puppet-cressex-lab.git
[deleted] new_branch
Advertisements

NetApp SnapMirror

 

NetApp SnapMirror is a pretty easy utility to use, but offers great power. NetApp snapview snapshots a LUN to a given destination either on a demand basis or a schedule. You can set this up via the UI or you can run commands via SSH. In the following example, we will be setting up snapview on a LUN.

However, there is one key concept here. You can dedicate different interfaces to handle different roles. So if you have your production data accessible on say e0a, then consider setting up your snapshot traffic on e0b. This way you do not flood the interface or the upstream network components. Also you are spreading the load.

Here’s a quick overview or some key concepts you need to consider:

We have a volume

ontap> vol status cloud01

Volume  State
cloud01 online

We have a lun

ontap> lun show

/vol/cloud01/cloud01-na-lun01    100g (107374182400)  (r/w,online,mapped)

And this is our satatus of SnapMirror

ontap> snapmirror status Snapmirror is on. 
ontap> snapmirror destinations snapmirror destination: no known destinations

We need to edit the /etc/snapmirror.conf file and place some information in ro configure a snapmirror job. Here is a good article on what options you can use. http://www.wafl.co.uk/tag/snapmirrorconf/

The following snapmirror.conf entry indicates that filer ontap1′s volume cloud04_mirror will mirror volume cloud04 via the ontap0-gig interface. The ontap0-gig interface is whatever IP address ontap1 can resolve that name to. In this case, it might be a gigabit ethernet link on filer ontap0. The mirror is updated at 9:30 a.m., 1:30 p.m., and 7:30 p.m., Monday through Friday. The asterisk means that the data replication schedule is not affected by the day of month; it is the same as entering numbers 1 through 31 (comma-separated) in that space. It is actually similar to a cron format.

ontap0-gig:cloud01 ontap1:cloud01_mirror – 30 9, 13, 19 * 1, 2, 3, 4, 5

The important part here is you can tell snapmirror.conf the interface name to send the snapmirror replication traffic down. You may also need to set the /etc/snapmirror.allow to allow snapview connetions, otherwise you may see a connection denied issue.

Here is the commands you need to look at

ontap0> wrfile /etc/snapmirror.allow 10.10.10.2 ontap1

This will configure ontap0 to allow snapmirror connections to ontap1. As long as the name resolution is set correctly, it will traverse the interface as generic IP routing will take care of the traffic flow.

The other iimportant thing you can consider is the bandwidth shaping. The dash (–) in the command above, which is actually at the argu_ments field location, indicates that both the kbs and restart arguments are set to default.

So what is the ksb?

Taken from the link above (I said it was good) is this extract:

The value for this argument specifies the maximum speed (in kilobytes per second) at which SnapMirror data is transferred over the network. The kbs setting is used to throttle network bandwidth consumed, disk I/O, and CPU usage. By default, the filer transfers the data as fast as it can. The throttle value is not used while synchronously mirroring.

You can set it something like this:

kbs=2000

This means the transfer speed is set at a maximum rate of 2, 000 kilobytes per second.

Asyn, Sync or semi sync

In async mode, snapshot copies of the volume are created periodically on the source. Only blocks that have changed or have been newly created since the last replication cycle are transferred to the target, making this method very efficient in terms of storage system overhead and network bandwidth.

Sync mode sends updates from the source to the destination as they occur, rather than according to a predetermined schedule. This helps data written on the source system to be protected on the destination even if the entire source system fails. NVLOG forwarding and consistency point (CP) forwarding are used to keep the target completely up to date. NVLOG forwarding enables data from the write log that is normally cached in NVRAM on NetApp storage to be synchronized with the target. Consistency point forwarding keeps the on-disk file system images synchronized.

Semi-sync mode differs from sync mode in two ways. Writes to the source aren’t required to wait for acknowledgement from the target before they are committed and acknowledged, and NVLOG forwarding is not used. These two changes speed up application response with only a very small hit in terms of achievable recovery point objective (RPO).

SnapMirror status?

We have can see the status through both the CLI and the UI. Here is the CLI

ontap0> snapmirror status
Snapmirror is on.
Source         Destination            State          Lag       Status
ontap0:cloud01 ontap1:cloud01_mirror  Snapmirrored   00:00:29  Idle

Here you can see the command snapmirror status shows the output of the status of the SnapMirror replication. Pretty simple right?

The UI shows a little more information as shown here:


Nice images 😦 but you get the drift.

There lots more to this and maybe I will add to this blog at a later date, but you can see how SnapMirror can provide real time data by using synchronous mirroring for your volumes. Also, using methods of traffic shaping and IP routing, you can traverse your snapmirror vols over a specific interface. Lots of good options. Enjoy!

Installing Python and Django

django

Here are my notes on how to set up Python and Django

UPDATE: These are old notes and I would definitely use virtenv when setting up Python. I will try to do a blog about this

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

# Depending on whether you run as root, you may need to use sudo.

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

sudo yum install gcc tcl tk sqlite-devel readline-devel gdbm-devel -y
sudo yum install tkinter ncurses-devel libdbi-devel tk-devel zlib-devel -y
sudo yum install openssl-devel bzip2-devel -y
sudo yum install httpd httpd-devel -y
sudo yum install mysql mysql-devel -y
wget http://www.python.org/ftp/python/2.7.3/Python-2.7.3.tgz
tar zxvf Python-2.7.3.tgz
cd Python-2.7.3
sudo ./configure --prefix=/opt/python2.7 --with-threads --enable-shared
make
sudo make install
ln -s /opt/python2.7/bin/python /usr/bin/python2.7
echo '/opt/python2.7/lib'>> /etc/ld.so.conf.d/opt-python2.7.conf
ldconfig
echo "alias python='/opt/python2.7/bin/python'" >> /etc/bashrc
echo "alias python2.7='/opt/python2.7/bin/python'" >> /etc/bashrc

***log out and log back in at this point. This is to ensure your bash rpofie is updated with the new python location***

wget http://pypi.python.org/packages/2.7/s/setuptools/setuptools-0.6c11-py2.7.egg
sh setuptools-0.6c11-py2.7.egg
wget http://modwsgi.googlecode.com/files/mod_wsgi-3.3.tar.gz
tar zxvf mod_wsgi-3.3.tar.gz
cd mod_wsgi-3.3
./configure --with-python=/opt/python2.7/bin/python
make
make install
curl http://python-distribute.org/distribute_setup.py | python
curl https://raw.github.com/pypa/pip/master/contrib/get-pip.py | python
pip install Django