Ec2 (aws) - delete snapshots

Ec2 snapshots are a way to make backups of your data into the amazon cloud. To do snapshots you will need the ec2-api-tools, your access key and secret or the x509 certificates for your aws account. Obviously after you snapshot you will need eventually to delete snapshots that you don't need anymore. This example shows how to use the ec2-api-tools into a shell to delete snapshots that are not part of the current month. You can have a cronjob that runs every last day of the month, this will give you almost 30 days of snapshots.

# describe snapshots and sort by date
ec2-describe-snapshots -C cert.pem  -K key.pem | sort -k 5

# delete all but current month (not the last 30 days)
ec2-describe-snapshots -C cert.pem  -K key.pem | grep -v $(date +%Y-%M-) |  awk '{print $2}' | xargs -n 1 -t ec2-delete-snapshot -K key.pem -C cert.pem

Couchbase recover web console password

You will need to have access to the config.dat file that resides onto the couchbase server (can be any of them if is into a cluster).

/opt/couchbase/bin/erl \
-noinput -eval \
'case file:read_file("/opt/couchbase/var/lib/couchbase/config/config.dat") of {ok, B}  -> io:format("~p~n", [binary_to_term(B)]) end.' \
-run init stop | grep cred
  {rest_creds,
         {creds,[{"Administrator",[{password,"Administrator"}]}]}]},

There you go
username : Administrator
password : Administrator

Puppet install rpms via http sources

The redhat package manager - rpm has the capability to install packages
from an url. As simple as

rpm -ivh http://example.com/package.rpm

Taking this in consideration we can use this into puppet to install packages
from an url.

Save this text as test.pp

class examplerpm ( $src ) {

  package { 'package':
     provider => 'rpm',
     ensure => installed,
     source => "${examplerpm::rpm}"
 }
}

class { 'examplerpm':
  src => 'https://example.com/package.rpm',
}


Apply the manifest with puppet

puppet apply --debug --no-daemonize test.pp

Voila - the package is installed via puppet->rpm provider.
The key to all this is to specify the provider into the Package section of
examplerpm class. This ensures that rpm will go fetch the source and installs
it.

How to calculate the MySQL database size

Connect to mysql and run the command bellow

# total db size
SELECT table_schema "Data Base Name", SUM( data_length + index_length) / 1024 / 1024 
"Data Base Size in MB" FROM information_schema.TABLES GROUP BY table_schema ;

# total per db size
SELECT TABLE_NAME, table_rows, data_length, index_length, 
round(((data_length + index_length) / 1024 / 1024),2) "Size in MB"
FROM information_schema.TABLES WHERE table_schema = "schema_name";

Tricks with nose and python

Nose is a very useful tool for running unittest in python. These are a few tricks you can use. Bellow is my test file - I called it test.py

# dummy case test 

class Test():

    def test_algo(self):
        assert 0 == 0, '0 is not equal to 0'

    def test_failed(self):
        print 'this will fail'
        assert 1 == 0, '0 is not equal to 1'

    def test_fail_inpdb(self):
        # div by 0
        1/0

Now let's see what is this about

1. first I use assert to check if the results match
2. based on assert the first function will pass and second will fail
3. the last function will trigger and Error not a Failure

# running with pdb so any Error not Failure will drop me into python debugger
$ nosetests  --pdb  test.py
.> /home/silviud/PROGS/PYTHON/wal/tests/test.py(14)test_fail_inpdb()
-> 1/0
(Pdb) l
  9             print 'this will fail'
 10             assert 1 == 0, '0 is not equal to 1'
 11     
 12         def test_fail_inpdb(self):
 13             # div by 0
 14  ->         1/0          #### this is the line that triggers the error
[EOF]
(Pdb) c
EF
======================================================================
ERROR: tests.test.Test.test_fail_inpdb
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/silviud/Environments/2.7/lib/python2.7/site-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/home/silviud/PROGS/PYTHON/wal/tests/test.py", line 14, in test_fail_inpdb
    1/0
ZeroDivisionError: integer division or modulo by zero

======================================================================
FAIL: tests.test.Test.test_failed
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/silviud/Environments/2.7/lib/python2.7/site-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/home/silviud/PROGS/PYTHON/wal/tests/test.py", line 10, in test_failed
    assert 1 == 0, '0 is not equal to 1'
AssertionError: 0 is not equal to 1
-------------------- >> begin captured stdout << ---------------------
this will fail

--------------------- >> end captured stdout << ----------------------

----------------------------------------------------------------------
Ran 3 tests in 8.453s

FAILED (errors=1, failures=1)

Now let's pretended that I run this regular and is part of my Continuous Integration server which so happen
to be running Jenkins. How can I integrate the python unittests with it ?!
Simple - nose has many plugins and one of them is xunit.

$ nosetests  --with-xunit test.py
....
$ cat nosetests.xml
<?xml version="1.0" encoding="UTF-8"?><testsuite name="nosetests" tests="3" errors="1" failures="1" skip="0"><testcase classname="tests.test.Test" name="test_algo" time="0.000" /><testcase classname="tests.test.Test" name="test_fail_inpdb" time="0.000"><error type="exceptions.ZeroDivisionError" message="integer division or modulo by zero"><![CDATA[Traceback (most recent call last):
  File "/usr/lib/python2.7/unittest/case.py", line 321, in run
    testMethod()
  File "/home/silviud/Environments/2.7/lib/python2.7/site-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/home/silviud/PROGS/PYTHON/wal/tests/test.py", line 14, in test_fail_inpdb
    1/0
ZeroDivisionError: integer division or modulo by zero
]]></error></testcase><testcase classname="tests.test.Test" name="test_failed" time="0.001"><failure type="exceptions.AssertionError" message="0 is not equal to 1&#10;-------------------- &gt;&gt; begin captured stdout &lt;&lt; ---------------------&#10;this will fail&#10;&#10;--------------------- &gt;&gt; end captured stdout &lt;&lt; ----------------------"><![CDATA[Traceback (most recent call last):
  File "/usr/lib/python2.7/unittest/case.py", line 321, in run
    testMethod()
  File "/home/silviud/Environments/2.7/lib/python2.7/site-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/home/silviud/PROGS/PYTHON/wal/tests/test.py", line 10, in test_failed
    assert 1 == 0, '0 is not equal to 1'
AssertionError: 0 is not equal to 1
-------------------- >> begin captured stdout << ---------------------
this will fail

--------------------- >> end captured stdout << ----------------------
]]></failure></testcase></testsuite>

Just by adding --with-xunit made nose to active the xunit plugin and it generated an xml file into nosetests.xml - this file can be used by Jenkins to take decisions if the build failed or not !

Shell parallel processing

This is the description of the tool from the gnu site - Parallel
GNU parallel is a shell tool for executing jobs in parallel using one
or more computers. A job it can be a single command or a small script
that has to be run for each of the lines in the input. The typical
input is a list of files, a list of hosts, a list of users, a list of
URLs, or a list of tables. A job can also be a command that reads from
a pipe. GNU parallel can then split the input into blocks and pipe a
lock into each command in parallel.

The tool can do many things and has some very useful tools that come with it
see sql and niceload.
Bellow you can see some example on how to use it.

#!/bin/sh

# tail log files on different computers
# create a hosts file with all the computers you want to connect

echo '10.100.218.79' >> host.file
echo '107.22.24.219' >> host.file
cat host.file | parallel ssh {} "tail /var/log/php-fpm/error.log | awk '{print \$1,\$2,\$3,\$4,\$5,\$6}'"

[20-Feb-2012 15:05:05.323178] DEBUG: pid 7812, fpm_pctl_perform_idle_server_maintenance(),
[20-Feb-2012 15:05:06.324028] DEBUG: pid 7812, fpm_pctl_perform_idle_server_maintenance(),
[20-Feb-2012 15:05:07.324877] DEBUG: pid 7812, fpm_pctl_perform_idle_server_maintenance(),
[20-Feb-2012 15:05:08.325727] DEBUG: pid 7812, fpm_pctl_perform_idle_server_maintenance(),
[20-Feb-2012 15:05:09.326568] DEBUG: pid 7812, fpm_pctl_perform_idle_server_maintenance(),
[20-Feb-2012 15:05:10.327418] DEBUG: pid 7812, fpm_pctl_perform_idle_server_maintenance(),
[20-Feb-2012 15:05:11.328265] DEBUG: pid 7812, fpm_pctl_perform_idle_server_maintenance(),
[20-Feb-2012 15:05:12.329118] DEBUG: pid 7812, fpm_pctl_perform_idle_server_maintenance(),
[20-Feb-2012 15:05:13.329960] DEBUG: pid 7812, fpm_pctl_perform_idle_server_maintenance(),
[20-Feb-2012 15:05:14.330806] DEBUG: pid 7812, fpm_pctl_perform_idle_server_maintenance(),

GNU's site has lots more example see Examples

Install cProfile on debian 6.0.3 (squeeze)

So you just seen this when tried to profile something on debian squeeze

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python2.6/cProfile.py", line 36, in run
    result = prof.print_stats(sort)
  File "/usr/lib/python2.6/cProfile.py", line 80, in print_stats
    import pstats
ImportError: No module named pstats

Well all is need to fix it is to enable a repository and then install python-profile

echo 'deb http://ftp.ca.debian.org/debian squeeze main non-free' >> /etc/apt/sources.list
# replace .ca. with your country code
apt-get update
aptitude install  python-profiler
python

>>> import cProfile
>>> def f():
...     print 'called'
...
>>> cProfile.run('f()')
called
         3 function calls in 0.000 CPU seconds

   Ordered by: standard name

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.000    0.000    0.000    0.000 :1(f)
        1    0.000    0.000    0.000    0.000 :1()
        1    0.000    0.000    0.000    0.000 {method 'disable' of '_lsprof.Profiler' objects}

MySQL backup in time

Run mysqldump on master database (needs to have innodb)

# replication point in time
mysqldump --single-transaction --flush-logs \
--master-data=2 --all-databases > backup.sql

# Note the position and the log file from the backup.sql and insert it into the slave</>

shell# mysql -u USER -p PASS
mysql> stop slave;
shell# mysql < backup.sql
shell# mysql -u USER -p PASS 
mysql> CHANGE MASTER TO MASTER_LOG_FILE='the_log_file_written_into_dump',
mysql> MASTER_LOG_POS = xxx ;
mysql> start slave;

Create storage infrastructure with libvirt

Libvirt is defacto library to manage virtual machines. This will show how you can create a storage pool that can be used later to allocate space for vms.

my storage space is located under /storage and the type is a regular directory as i want to store image files in there. this is low performance but will do for my dev machines.

first you need to create the directory

mkdir /storage

# then you need to have an xml file as follow - see type is dir and path is /storage

<pool type="dir">
        <name>storage_01</name>
        <target>
          <path>/storage</path>
        </target>

</pool>

# than go into virsh and define the new pool - define will make the pool  persistent into the /etc/libvirt/

virsh# pool-list
virsh# pool-autostart pool_name
<name>sparse.img</name>
<allocation>0</allocation>
<capacity unit="T">1</capacity>
<target>
  <path>/var/lib/virt/images/sparse.img</path>
  <permissions>
    <owner>107</owner>
    <group>107</group>
    <mode>0744</mode>
    <label>virt_image_t</label>
  </permissions>
</target>
</volume>

EC2 raid10 for mongo db

Running mongo db on a raid10(software raid) into ec2 is done via the ebs volumes. I'll show you how to

• create the raid10 on 8 ebs volumes
• (re) start the mdadm on the raid device
• mount the raid10 device and start using

I didn't use any config files for the raid devices so you will need to know how the devices are mapped and what uuid has the raid10.

Initial Creation of the raid

# you will need to have your ebs volumes attached to the server
mdadm --create --verbose /dev/md0 --level=10 --raid-devices=8 /dev/sdj /dev/sdk /dev/sdl /dev/sdm /dev/sdn /dev/sdo /dev/sdp /dev/sdq

# now create a file system 
mkfs.xfs /dev/md0

#mount the drive
mount /dev/md0 /mnt/mongo/data


# Obtain information about the array
mdadm --detail /dev/md0 # query detail

/dev/md0:
        Version : 0.90
  Creation Time : Wed Oct 26 19:37:16 2011
     Raid Level : raid10
     Array Size : 104857344 (100.00 GiB 107.37 GB)
  Used Dev Size : 26214336 (25.00 GiB 26.84 GB)
   Raid Devices : 8
  Total Devices : 8
Preferred Minor : 0
    Persistence : Superblock is persistent

    Update Time : Mon Dec 12 15:56:48 2011
          State : clean
 Active Devices : 8
Working Devices : 8
 Failed Devices : 0
  Spare Devices : 0

         Layout : near=2
     Chunk Size : 64K

           UUID : 144894cd:3b083374:1fa88d23:e4200572
         Events : 0.30

    Number   Major   Minor   RaidDevice State
       0       8      144        0      active sync   /dev/sdj
       1       8      160        1      active sync   /dev/sdk
       2       8      176        2      active sync   /dev/sdl
       3       8      192        3      active sync   /dev/sdm
       4       8      208        4      active sync   /dev/sdn
       5       8      224        5      active sync   /dev/sdo
       6       8      240        6      active sync   /dev/sdp
       7      65        0        7      active sync   /dev/sdq

# note the UUID and the devices
# Start the mongo database
/etc/init.d/mongod start

Shutdown(reboot) the server 

# restart the array device - you need to have the ebs volumes re-attached!
mdadm -Av /dev/md0 --uuid=144894cd:3b083374:1fa88d23:e4200572  /dev/sd*
mdadm: looking for devices for /dev/md0
mdadm: cannot open device /dev/sda1: Device or resource busy
mdadm: /dev/sda1 has wrong uuid.
mdadm: cannot open device /dev/sdb: Device or resource busy
mdadm: /dev/sdb has wrong uuid.
mdadm: cannot open device /dev/sdc: Device or resource busy
mdadm: /dev/sdc has wrong uuid.
mdadm: cannot open device /dev/sdr: Device or resource busy
mdadm: /dev/sdr has wrong uuid.
mdadm: cannot open device /dev/sds: Device or resource busy
mdadm: /dev/sds has wrong uuid.
mdadm: /dev/sdj is identified as a member of /dev/md0, slot 0.
mdadm: /dev/sdk is identified as a member of /dev/md0, slot 1.
mdadm: /dev/sdl is identified as a member of /dev/md0, slot 2.
mdadm: /dev/sdm is identified as a member of /dev/md0, slot 3.
mdadm: /dev/sdn is identified as a member of /dev/md0, slot 4.
mdadm: /dev/sdo is identified as a member of /dev/md0, slot 5.
mdadm: /dev/sdp is identified as a member of /dev/md0, slot 6.
mdadm: /dev/sdq is identified as a member of /dev/md0, slot 7.
mdadm: added /dev/sdk to /dev/md0 as 1
mdadm: added /dev/sdl to /dev/md0 as 2
mdadm: added /dev/sdm to /dev/md0 as 3
mdadm: added /dev/sdn to /dev/md0 as 4
mdadm: added /dev/sdo to /dev/md0 as 5
mdadm: added /dev/sdp to /dev/md0 as 6
mdadm: added /dev/sdq to /dev/md0 as 7
mdadm: added /dev/sdj to /dev/md0 as 0
mdadm: /dev/md0 has been started with 8 drives.

# now you can mount the array
mount /dev/md0 /mnt/mongo/data/

# start the mongo database
/etc/init.d/mongod start

Apache rewrite rule to redirect to https

Problem - you want to redirect all http traffic to https.
The following rewrite rule will redirect any web site that
you are running so is no need to write the server name.

# into httpd.conf write the following

RewriteEngine On
RewriteCond %{HTTPS} off
RewriteRule (.*) https://%{HTTP_HOST}%{REQUEST_URI}

# then restart apache 
# this requires a working version of https on the same web server

Howto create an AMI from a running instance into Ec2 cli

In order to create an ami from an EC2 running instance you will need.

• certificate file from your aws account credentials
• private key for the cerificate file from your aws account credentials(you can download this only at certificate creation)
• access by ssh to your running instance
• access key for AWS
• access secret key for AWS
• any ec2 tools - I used amitools

# create the bundle under /mnt
ec2-bundle-vol -d /mnt -k /root/key.pem -c /root/cer.pem -u xxxxxxxxxxxx
# xxxxxxxxxxxx is your account number without dashes
ec2-upload-bundle -b YOURBUCKET -m /mnt/image.manifest.xml -a YOUR_ACCESS_KEY -s YOUR_ACCESS_SECRET_KEY
# register the ami so is available 
ec2-register -K /root/key.pem -C /root/cer.pem -n SERVER_NAME YOURBUCKET/image.manifest.xml
# this will respond with something like 
IMAGE   ami-xxxxxxxx

# At this point you can go into the aws console and boot a new instance from the ami you registered.<br />
# to deregister the ami 
ec2-deregister  ami-xxxxxxxx

From domU read the xenstore (ec2, linode etc)

In case you wonder what is the dom0 running for your instance/vps this will give you information from xenstore. Taken from a FreeBSD receipe and adapted to linux.

Building & installation
-----------------------

Prerequisites: make, XENHVM or XEN kernel (GENERIC will not work) - all this is already there if you run as pv.

1. wget http://bits.xensource.com/oss-xen/release/4.1.1/xen-4.1.1.tar.gz

2. tar xvfz xen-4.1.1.tar.gz

3. cd xen-4.1.1/tools

4. make -C include

5. cd misc

6. make xen-detect

7. install xen-detect /usr/local/bin

8. cd ../xenstore

9. Build client library and programs:
  make clients

10. Install client library and programs:
  install libxenstore.so.3.0 /usr/local/lib
  install xenstore xenstore-control /usr/local/bin
  cd /usr/local/bin
  ln xenstore xenstore-chmod
  ln xenstore xenstore-exists
  ln xenstore xenstore-list
  ln xenstore xenstore-ls
  ln xenstore xenstore-read
  ln xenstore xenstore-rm
  ln xenstore xenstore-write

(in case that your ld loader doesn't look into /usr/local/lib do this
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/lib)


Usage
-----

1. Set required environment variable:
  export XENSTORED_PATH=/dev/xen/xenstore -- FreeBSD
  export XENSTORED_PATH=/proc/xen/xenbus -- Linux

2. Now you can do things such as:
  xen-detect
  xenstore-ls device
  xenstore-ls -f /local/domain/0/backend/vif/11/0
  xenstore-read name

Ec2 metadata

In case that you are looking for more info while you are into a ec2 instance you can call
from within the instance the api metadata server from ec2.

$ curl http://169.254.169.254/latest/meta-data/
ami-id
ami-launch-index
ami-manifest-path
block-device-mapping/
hostname
instance-action
instance-id
instance-type
kernel-id
local-hostname
local-ipv4
mac
network/
placement/
profile
public-hostname
public-ipv4
public-keys/
ramdisk-id
reservation-id

MySQL cluster (ndb engine) setup

Setup Mysql cluster with the NDB engine

Mysql Cluster is a high availability RDBMS that can be
downloaded from mysql.com.
It uses as a storage engine NDB which is an engine developed
by Ericsson based on shared nothing architecture.

The roles into the setup are split as follow:

- mysql server (this is a pure mysql server configured with the
engine NDB)
- data nodes (this is a storage node that runs ndbd daemon)
- management server (this will orchestrate all the actions into
the cluster)

My setup
mgmt_node    : 192.168.149.128
data_node_1  : 192.168.149.130
mysql_server : 192.168.149.131

[Installation]

mgmt_node
- install client and management rpms
128#rpm -ivh MySQL-Cluster-gpl-client-7.1.15-1.rhel5.i386.rpm
128#rpm -ivh MySQL-Cluster-gpl-management-7.1.15-1.rhel5.i386.rpm
128#rpm -ivh MySQL-Cluster-gpl-tools-7.1.15-1.rhel5.i386.rpm

data_node_1
- install storage
130#rpm -ivh MySQL-Cluster-gpl-storage-7.1.15-1.rhel5.i386.rpm

mysql_server
- install mysql server
131#rpm -ivh MySQL-Cluster-gpl-server-7.1.15-1.rhel5.i386.rpm
131#rpm -ivh MySQL-Cluster-gpl-client-7.1.15-1.rhel5.i386.rpm

[Configuration]

mgmt_node
- configure location
128# mkdir /mysql-cluster
- configuration file

128# cat > /mysql-cluster/config.ini << CONFIG
[NDBD DEFAULT]
NoOfReplicas=1
DataMemory=20M
IndexMemory=10M

[TCP DEFAULT]
portnumber=1186

[NDB_MGMD]
hostname=192.168.149.128
datadir=/mysql-cluster

# repeat this with the number of data nodes into the cluster
[NDBD]
hostname=192.168.149.130
datadir=/mysql-cluster/data

[MYSQLD]
hostname=192.168.149.131

CONFIG


data_node_1
- configure location
130# mkdir -p /mysql-cluster/data
- configuration file
130# cat > /etc/my.cnf << CONFIG
[MYSQLD]
ndbcluster
ndb-connectstring=192.168.149.128

[MYSQL_CLUSTER]
ndb-connectstring=192.168.149.128

CONFIG

mysql_server
- configure location
131# mkdir -p /mysql-cluster/data
- configuration file
131# cat > /etc/my.cnf << CONFIG
[MYSQLD]
ndbcluster
ndb-connectstring=192.168.149.128

[MYSQL_CLUSTER]
ndb-connectstring=192.168.149.128

CONFIG


[Startup]

mgmt_node
128#ndb_mgmd --initial -f /mysql-cluster/config.ini
MySQL Cluster Management Server mysql-5.1.56 ndb-7.1.15
2011-08-18 07:09:50 [MgmtSrvr] INFO     -- The default config directory
'/usr/mysql-cluster' does not exist. Trying to create
it...
2011-08-18 07:09:50 [MgmtSrvr] INFO     -- Sucessfully created config
directory
2011-08-18 07:09:50 [MgmtSrvr] WARNING  -- at line 7: [TCP] portnumber is
deprecated

data_node_1
130#ndbd --initial
Unable to connect with connect string: nodeid=0,192.168.149.128:1186
Retrying every 5 seconds. Attempts left: 12 11 10 9 8 7 6 5
2011-08-18 07:11:44 [ndbd] INFO     -- Angel connected to
'192.168.149.128:1186'
2011-08-18 07:11:44 [ndbd] INFO     -- Angel allocated nodeid: 2

mysql_node
131#/etc/init.d/mysql start
Starting MySQL.... SUCCESS!


[Running Operations]

128#ndb_mgm
ndb_mgm> show
Cluster Configuration
---------------------
[ndbd(NDB)]     1 node(s)
id=2    @192.168.149.130  (mysql-5.1.56 ndb-7.1.15, Nodegroup: 0, Master)

[ndb_mgmd(MGM)] 1 node(s)
id=1    @192.168.149.128  (mysql-5.1.56 ndb-7.1.15)

[mysqld(API)]   1 node(s)
id=3    @192.168.149.131  (mysql-5.1.56 ndb-7.1.15)

Cisco ace - virtualization

Ace is a load balancer from cisco systems - see http://www.cisco.com/en/US/products/ps6906/index.html for details

How to verify the virtualization options:

  show running-config context
  show running-config domain
  show running-config resource-class
  show running-config role

Configure a context

  host1/Admin# config
  (config)#
  host1/Admin(config)# context C1 # Creates a context & enter configuration mode.
  host1/Admin(config-context)  
  host1/Admin(config)# no context C1 # Deletes context
  host1/Admin(config-context)# do copy running-config startup-config # save config

Moving between contexts

  host1/Admin# changeto C1 # Change onto C1 context
  host1/C1#
  host1/C1# exit # exit from context
  show service-policy summary |i IP # will give you what policies are available for that ip
  show probe |i ip|port will tell you what probes are on

Quick start with GlusterFS

The software that www.gluster.org makes allows to have distributed file systems with commodity hardware.

Rpm
===

http://download.gluster.com/pub/gluster/glusterfs/LATEST/CentOS/glusterfs-fuse-3.2.0-1.x86_64.rpm
http://download.gluster.com/pub/gluster/glusterfs/LATEST/CentOS/glusterfs-core-3.2.0-1.x86_64.rpm
http://download.gluster.com/pub/gluster/glusterfs/LATEST/CentOS/glusterfs-rdma-3.2.0-1.x86_64.rpm


Gluster Daemon
==============

/etc/init.d/glusterd start


Firewall
========

iptables -A RH-Firewall-1-INPUT -m state --state NEW -m tcp -p tcp --dport 24007:24008 -j ACCEPT
iptables -A RH-Firewall-1-INPUT -m state --state NEW -m tcp -p tcp --dport 24009:24014 -j ACCEPT


Bricks
======

gluster peer probe SERVER_NAME_OR_IP


Volumes
=======

    Creation
    ========
    # replicated (mirror onto 2 hosts)
    gluster volume create test-volume replica 2 transport tcp server1:/exp1 server2:/exp2

    Start
    =====
    gluster volume start test-volume

    List
    ====
    gluster volume info (all|vol_name)

    Mount
    =====
    mount -t glusterfs HOSTNAME-OR-IPADDRESS:/VOLNAME MOUNTDIR

Ssh execute remote commands

Ssh is a very very useful tool and some tricks with it will help you a lot of time and typing. For example I want to transfer a file remote like my ssh-key file. I have a few options:

• use scp to transfer the file, then login into the remote system and execute the commands.
  
• use a different mechanism to transfer the files(ftp etc), login and execute the commands.
  
• do it all in one command ... this is the cool one - see bellow
  

shell$ cat ~sd/.ssh/id_rsa.pub | ssh root@192.168.0.105 'cat - > .ssh/authorized_keys'
shell$ cat ~sd/.ssh/id_rsa.pub | ssh root@192.168.0.105 'cat - > .ssh/authorized_keys2'

# usually the authorized_keys is a link to authorized_keys2 but in this I just have two separate files.
# as you can see is nothing else to do :)

Am I hacked ?

You do a ps -ef and you think is all good ... but perhaps what you see is not exactly what is really running ... This is a simple but effective way to compare the running processes reported by ps with what is into /proc

shell$ ps ax | wc -l
shell$ 30
shell$ ls -d /proc/* | grep [0-9]|wc -l
shell$ 31 # there is one extra root kit perhaps :)

What happens when you do kill a program in linux ?

I had two simple questions:

• 
  q1: how to you stop(kill) a program in linux ?
  
• 
  a1: i use the kill command as in

  kill 99 or kill -9 99

• 
  q2: ok ... so what does it really happens ?
  
• 
  a2: himm ... good question - well i send a signal to the program via a system call and then the kernel will take care of the rest ... as in will kill the program
  q2.1: himm so how does it kill it ?! what does it really happens
  a2.1: you know what let me think about it ... yeah i didn't look into this - well let me trace it and will find out ...
  

  shell$ bash &
  [2] 29120
  shell$ strace kill 29120
  execve("/usr/bin/kill", ["kill", "29120"], [/* 23 vars */]) = 0
  brk(0)                                  = 0x8849000
  access("/etc/ld.so.preload", R_OK)      = -1 ENOENT (No such file or directory)
  open("/etc/ld.so.cache", O_RDONLY)      = 3
  fstat64(3, {st_mode=S_IFREG|0644, st_size=24036, ...}) = 0
  mmap2(NULL, 24036, PROT_READ, MAP_PRIVATE, 3, 0) = 0xb7f1e000
  close(3)                                = 0
  open("/lib/libc.so.6", O_RDONLY)        = 3
  read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\340\17K\0004\0\0\0"..., 512) = 512
  fstat64(3, {st_mode=S_IFREG|0755, st_size=1611564, ...}) = 0
  mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb7f1d000
  mmap2(0x49b000, 1332676, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x110000
  mprotect(0x24f000, 4096, PROT_NONE)     = 0
  mmap2(0x250000, 12288, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x13f) = 0x250000
  mmap2(0x253000, 9668, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x253000
  close(3)                                = 0
  mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb7f1c000
  set_thread_area({entry_number:-1 -> 6, base_addr:0xb7f1c6c0, limit:1048575, seg_32bit:1, contents:0, read_exec_only:0, limit_in_pages:1, seg_not_present:0, useable:1}) = 0
  mprotect(0x250000, 8192, PROT_READ)     = 0
  mprotect(0x497000, 4096, PROT_READ)     = 0
  munmap(0xb7f1e000, 24036)               = 0
  brk(0)                                  = 0x8849000
  brk(0x886a000)                          = 0x886a000
  kill(29120, SIGTERM)                    = 0
  exit_group(0)                           = ?
  
  

  from the second line at the bottom i can see that a kill(PID, SIGTERM) was sent to the process and the return code is 0 (meaning success), but does it really happens into the kernel ?! - it will take me a lot more to explain but I found a good article about it at http://www.ibm.com/developerworks/library/l-linux-process-management/