Showing posts with label sysadmin. Show all posts
Showing posts with label sysadmin. Show all posts

Monday, February 18, 2013

Ansible within AWS (ec2)

Ansible is a new configuration/orchestration management framework and is just awesome!

Why is that ?

  • very short learning curve
  • no need for a specific data service language
  • can be used to both execute/configure machines
  • very simple to write your own modules
  • can be used into a push or pull model
  • ... ansible.cc ... for more info

This is how you can use it within aws(ec2) to manage services.

# Install ansible via git
$ cd /tmp
$ git clone https://github.com/ansible/ansible.git
$ cd ansible
$ python setup.py install
$ pip install boto # used for the ec2 inventory

# setup aws variables
$ export ANSIBLE_HOSTS=/tmp/ansible/plugins/inventory/ec2.py # ec2 inventory
$ export AWS_ACCESS_KEY_ID='YOUR_AWS_API_KEY'
$ export AWS_SECRET_ACCESS_KEY='YOUR_AWS_API_SECRET_KEY'

# setup ssh access
$ ssh-agent 
SSH_AUTH_SOCK=/tmp/ssh-dFUXvhH31724/agent.31724; export SSH_AUTH_SOCK;
SSH_AGENT_PID=31725; export SSH_AGENT_PID;
echo Agent pid 31725;
$ ssh-add /PATH_TO/YOUR_SSH_KEY_OR_PEM

# I use ec2-user onto a amazon linux
ansible -m ping all -u ec2-user
ec2-54-242-33-49.compute-1.amazonaws.com | success >> {
    "changed": false, 
    "ping": "pong"
}

The ec2.py inventory has connected to the aws api and obtained all the instances running within the account that has the exported credentials AWS SECRET/KEY. Then ansible used the ping module -m ping to ping the host(s). The ping module just connects via ssh to a host and reports pong with changed: false.

Now that we can connect let's see if we can leverage some of the metadata offered by AWS. My server runs into the security group ssh-web and to access this information from within ansible all you have to do is to use security_group_ssh-web. Where this come from is the ec2.py inventory script, if you run the script directly you will see something like this.

$ /tmp/ansible/plugins/inventory/ec2.py

{
  "i-e4c9ca9c": [
    "ec2-54-242-33-49.compute-1.amazonaws.com"
  ], 
  "key_mykey": [
    "ec2-54-242-33-49.compute-1.amazonaws.com"
  ], 
  "security_group_ssh-web": [
    "ec2-54-242-33-49.compute-1.amazonaws.com"
  ], 
  "tag_Name_srv01": [
    "ec2-54-242-33-49.compute-1.amazonaws.com"
  ], 
  "type_t1_micro": [
    "ec2-54-242-33-49.compute-1.amazonaws.com"
  ], 
  "us-east-1": [
    "ec2-54-242-33-49.compute-1.amazonaws.com"
  ], 
  "us-east-1b": [
    "ec2-54-242-33-49.compute-1.amazonaws.com"
  ]
}

In order to start the apache web server on all instances belonging to the ssh-web group is as simple as:

ansible -m service -a "name=httpd state=started"  security_group_ssh-web  -u ec2-user -s
ec2-54-242-33-49.compute-1.amazonaws.com | success >> {
    "changed": true, 
    "name": "httpd", 
    "state": "started"
}

# notice -s which stands for use sudo without password 
From here on sky is the limit, you can take a look at the docs site http://ansible.cc/docs/ for more complex examples.

Tuesday, October 30, 2012

Ec2 (aws) - delete snapshots

Ec2 snapshots are a way to make backups of your data into the amazon cloud. To do snapshots you will need the ec2-api-tools, your access key and secret or the x509 certificates for your aws account. Obviously after you snapshot you will need eventually to delete snapshots that you don't need anymore. This example shows how to use the ec2-api-tools into a shell to delete snapshots that are not part of the current month. You can have a cronjob that runs every last day of the month, this will give you almost 30 days of snapshots.
# describe snapshots and sort by date
ec2-describe-snapshots -C cert.pem  -K key.pem | sort -k 5

# delete all but current month (not the last 30 days)
ec2-describe-snapshots -C cert.pem  -K key.pem | grep -v $(date +%Y-%M-) |  awk '{print $2}' | xargs -n 1 -t ec2-delete-snapshot -K key.pem -C cert.pem

Friday, October 19, 2012

Couchbase recover web console password

You will need to have access to the config.dat file that resides onto the couchbase server (can be any of them if is into a cluster).
/opt/couchbase/bin/erl \
-noinput -eval \
'case file:read_file("/opt/couchbase/var/lib/couchbase/config/config.dat") of {ok, B}  -> io:format("~p~n", [binary_to_term(B)]) end.' \
-run init stop | grep cred
  {rest_creds,
         {creds,[{"Administrator",[{password,"Administrator"}]}]}]},
There you go
username : Administrator
password : Administrator

Thursday, November 11, 2010

Port redirect in linux to remote host

I had an application that needs to connect to a LDAP port (tcp:389) but the problem was that the LDAP was on an external network and on a non standard port (tcp:1389).
Pointing direct to remote_ip:1389 was not and option because the application is taking the LDAP port from the openldap libs (LDAP_PORT) which is a constant integer = 389 ...

The solution proved to be very simple - install a small port redirect program rinted do a small configuration into .

# config
/etc/rinetd.conf
   
127.0.0.1     389        remote_ip        1389


# start the daemon (only if you install it from rpm - if not just start manually)
/etc/init.d/rinetd start



And this is it - all works just fine.

Note that doing an iptables PREROUTING and DNAT will not work in my case because iptables can do
redirects ONLY into the local network.

Monday, May 3, 2010

Xen resize disk (image) into DomU

The objective is to increase the storage allocated to a xen DomU vm.
Let's say the situation before we do anything looks like this:

The virtual machine name (as seen into xm list)  rhel-5.3-HARMONY
The vm is using tapio disk image files (not lvm) for the hard drive.

root@localhost.localdomain[~]11:12:00# df -k

Filesystem 1K-blocks Used Available Use% Mounted on

/dev/xvda 9048256 3198968 5481516 37% /
tmpfs 1048576 0 1048576 0% /dev/shm

I want to add an extra 1GB space to this the 9GB already in there (the red label above).

Steps to do this:
  • Shutdown the virtual machine (vm) - xm shutdown rhel-5.3-HARMONY
  • Make a backup of the disk  - cp rhel-5.3-HARMONY.img rhel-5.3-HARMONY.img.backup
  • Create an empty file with the desired size (I choosed 1GB) - dd if=/dev/zero of=ZeroContentFile bs=1024 count=1000000
  • Append the new empty file to the initial image file - cat ZeroContentFile >> rhel-5.3-HARMONY.img -  before we did the append on the filesystem (doing ls -al):
          -rwxr-xr-x 1 root root 9414967296 May 3 11:19 rhel-5.3-HARMONY.img
          and after:
          -rwxr-xr-x 1 root root 10438967296 May 3 11:24 rhel-5.3-HARMONY.img
  • Delete the temporary empty file - rm ZeroContentFile
  • Resize the image file - resize2fs -f rhel-5.3-HARMONY.img
  • Check with fsck the file - fsck.ext3 rhel-5.3-HARMONY.img
  • Start the vm to see if the new disk has been added - xm create /etc/xen/rhel-5.3-HARMONY.cfg
  • Check the disk space and you will see
root@localhost.localdomain[~]11:31:48# df -k
Filesystem 1K-blocks Used Available Use% Mounted on

/dev/xvda 10033864 3198928 6529108 33% /
tmpfs 1048576 0 1048576 0% /dev/shm


That's it you just increased your space by 1GB for the vm.
One thing to keep in mind is to do the backup of the original disk (just in case something goes wrong).

Monday, March 8, 2010

NFS server setup (getport(nfs): request from unauthorized host| dump(): request from unauthorized host ) RedHat EL

I tried to make a simple nfs server on a redhat el 5.4 that uses /etc/hosts.allow and /etc/hosts.deny

I have the following nfs setup:

server

into /etc/hosts
10.0.0.2  client
10.0.0.1 server

 into /etc/exports
 /home/nfs-server client(rw,no_root_squash)

into /etc/hosts.deny

# wildcard that denies all
ALL:ALL

into /etc/hosts.allow

mountd: 10.0.0.2
statd: 10.0.0.2
portmap: 10.0.0.2
rquotad: 10.0.0.2

I start the portmap (service portmap start) on both machines and try to mount the server from client ... by my surprise there is no luck && try to do a rpcinfo -p  - it failed with - No remote programs registered
Looking into the server logs I can see

portmap[3058]: connect from x.x.x.x  to getport(nfs): request from unauthorized host

portmap[3061]: connect from x.x.x.x to dump(): request from unauthorized host


Doing different searches on the subject I came accross a bug redhat has on their site
https://bugzilla.redhat.com/show_bug.cgi?id=465412

So ... the solution seems to be the following - you need to add the ip address of the client AND the client host name into /etc/hosts.allow - even though portmap has the tcp_wrappers / libwrap compiled static and doesn't read directly the files /etc/hosts.allow|deny.

This is what I added on the server /etc/hosts.allow


mountd: client
statd: client
portmap: client
rquotad: client


After this - happy NFS.

Disable at runtime selinux

There are times when you need to test something quick and selinux is in your way ... what do you do then ?
Instead of going with a full reboot you can just do the following

echo 0 > /selinux/enforce

This will disable the selinux at runtime. If the system is configured with selinux enabled into his config file
(on redhat/centos /etc/sysconfig/selinux ) next time you reboot it will be enable.

To enable at runtime

echo 1 > /selinux/enforce