Pomme::TAB

Life cycle of Rija Ménagé's random thoughts.

Getting RubyMotion Working on a Fresh Install of Mac OS X Lion

On Wednesday, I wanted to start hacking with RubyMotion. Because I was about to leave for a trip to HK, I didn’t try it on my Mountain Lion (10.8) running iMac and instead resurrected an old Snow Leopard (10.6) running Macbook.

A quick read through RubyMotion’s getting started guide and Apple’s Developer portal made me realize that I needed XCode 4.5 in order to easily install the command line tools (autoconf, make, git, …) and in order to use iOS SDK 6.

XCode 4.5 requires at least Mac OS X Lion (10.7), so had to upgrade the laptop (the Mac OS X Lion upgrade is not available anymore on the Mac App Store since Moutain Lion release and my Macbook is not supported by Mountain Lion, but I had thankfully kept on the iMac, a Mac OS X Lion installer I could re-use).

Once I got Lion,  Xcode 4.5 and RubyMotion installed, I went through the Hello World example and encountered a few problems along the way.

First problem:

undefined method `count’ for [“./app/app_delegate.rb”]:Array

This happen when running rake and after googling it turns it was because Ruby Motion requires at least ruby 1.8.7 and Lion comes with 1.8.6. I therefore decide to install and use RVM to install newer rubies which led to the second problem.

Second problem:

Readline related compilation error when installing any version of ruby using RVM.

After following several leads that didn’t work for my setup, I finally found two blogs mentioning an approach that actually works:

This allowed me to install ruby 1.9.* and I subsequently got further with the Hello World problem, straight into problem no. 3.

Third Problem:

After compiling the Hello app, rake hangs while launching the iOS simulator. The simulator is visible but is frozen.

It turns out it was because I use tmux. If I re-run rake in new terminal, not in tmux, it works fine.

To get rake working with tmux, I had to install reattach-to-user-namespace using Homebrew and in my tmux session, instead of calling rake directly, I use:

 reattach-to-user-namespace -l rake

It fixes the issue.

Fourth Problem:

CODESIGN_ALLOCATE error about ambiguous matching certificates is thrown by ruby motion when running rake device.

a post in the RubyMotion google group led to that Apple Tech. note that immediately helped: I had an older expired iOS developer certificate in my Mac OS X keychain. Removing it fixed the problem.

After all of that, RubyMotion now works for me.

Installing Puppet on Mac OS X

 

Puppet

Puppet is an open source (with enterprise version and support available)  client/server tool from Puppet Labs to facilitate the configuration and management of computer systems.

 

Written in ruby and available on many platforms, it offers a DSL that allows the “programming” of operational tasks across many machines.

The DSL covers abstracting the computer resources in a extensible way, and providing structure like class, modules and graphs that a configuration language can manipulate.

The server is called puppet master. It controls  a puppet agent installed on client machines that are to be managed. In addition of master and agent, there are puppet apply for stand alone use and puppet resource for accessing the Puppet resource abstraction layer.

Its extensibility makes it future proof  and there are providers (implements a resource abstraction in Puppet) for many platforms like Virtual box VMs and Amazon EC2.

The communication between clients and server is secured using SSL certificates.

This post is mainly for me so I can remember how I did install Puppet on Mac OS X and to allow me to repeat it on many mac systems.

The install  basically boils down to running a script I put on Gist (assuming you want to install Puppet 2.7.11 with Facter 1.6.6 on a startup disk called Macintosh HD):

bash -s 1.6.6 2.7.11 /Volumes/Macintosh\ HD < <(curl -s https://raw.github.com/gist/1895594/install_puppet_mac.sh)

The remainder of the post describes the gory details. I’ll keep this post updated as I learn more about the idiosyncrasies of Puppet on Mac.

Deployment environment

I’ve tested these instructions on Mac OS X Lion (10.7.3)  and Mac OS X Snow Leopard (10.6.8).

Downloading necessary files

For Mac OS X, there is a .pkg available for Facter and Puppet downlable from Puppet Lab web site:

http://downloads.puppetlabs.com/mac/

There are several versions available on that site.

For the purpose of this post, we will consider version 1.6.6 and 2.7.11 for Facter and Puppet respectively.

 

Puppet depends on Facter, so you need both. These are Mac OS X package on the site above.

There is another blog post [1] describing a way to install Puppet from source but the source links didn’t work for me when I tried.

Installation steps

Unpack the dmg

hdiutil attach facter-1.6.6.dmg
hdiutil attach puppet-2.7.11.dmg

Install the pkg

sudo installer -package /Volumes/facter-1.6.6/facter-1.6.6.pkg -target /Volumes/Macintosh\ HD
sudo installer -package /Volumes/puppet-2.7.11/puppet-2.7.11.pkg -target /Volumes/Macintosh\ HD

Create the puppet group and user

In other systems, the packaging may include the creation of the necessary puppet user and group.

 

The packages for Mac OS X don’t do that. Although it’s possible to create these when starting the puppet master with the –mkusers options, I prefer create then before hand during installation.

 max_gid=$(dscl . -list /Groups gid | awk '{print $2}' | sort -ug | tail -1)
 new_gid=$((max_gid+1))
 dscl . create /Groups/puppet
 dscl . create /Groups/puppet gid $new_gid




 max_uid=$(dscl . -list /Users UniqueID | awk '{print $2}' | sort -ug | tail -1)
 new_uid=$((max_uid+1))
 dscl . create /Users/puppet
 dscl . create /Users/puppet UniqueID $new_uid
 dscl . -create /Users/puppet PrimaryGroupID $new_gid

Create directories

mkdir -p /var/lib/puppet
mkdir -p /etc/puppet/manifests
mkdir -p /etc/puppet/ssl

Change permission on directories

chown -R puppet:puppet  /var/lib/puppet
chown -R puppet:puppet  /etc/puppet

Create puppet.conf

There are several sections, each relevant only to different puppet subcommands except for [main] which is global.

 

If puppet is running on a client ensure the server property is set to the machine running puppet master instead of local hostname as here.

echo "[main]
pluginsync = false
server = `hostname`

[master]
vardir = /var/lib/puppet
libdir = $vardir/lib
ssldir = /etc/puppet/ssl
certname = `hostname`




[agent]vardir = /var/lib/puppetlibdir = $vardir/libssldir = /etc/puppet/sslcertname = `hostname`




" > /etc/puppet/puppet.conf

Putting it all together

As I needed to install Puppet on more than one mac, I’ve made a script, inspired by trevmex’s tutorial [1], with all the steps together:

bash -s 1.6.6 2.7.11 /Volumes/Macintosh\ HD < <(curl -s https://raw.github.com/gist/1895594/install_puppet_mac.sh)

(Some of the steps uses sudo, so login password will be asked)

 

In the example above  I’ve  passed the version of Facter, the version of Puppet and my system disk as parameter to the script.

 

Create a hello world puppet class

Create the file /etc/puppet/manifests/site.pp with the following content:

import "nodes"

Create the file /etc/puppet/manifests/nodes.pp with the following content:

node default {




  notify {"Hello World":;}




}

Run the example:

puppet apply /etc/puppet/manifests/site.pp

you should see something like:

notice: Hello world




notice: /Stage[main]//Node[default]/Notify[Hello world]/message: defined 'message' as 'Hello world'




notice: Finished catalog run in 0.02 seconds

Setting up puppet in client/server mode

Testing on the same computer

Run the  puppet master

sudo puppet master --verbose --no-daemonize

You should see something like:

notice: Starting Puppet master version 2.7.11

Run the puppet agent in a separate shell by typing:

sudo puppet agent --test --server=`hostname`

You should see something like:

info: Caching catalog for wall-e.home




info: Applying configuration version '1330800282'




notice: Hello world




notice: /Stage[main]//Node[default]/Notify[Hello world]/message: defined 'message' as 'Hello world'




info: Creating state file /var/lib/puppet/state/state.yaml




notice: Finished catalog run in 0.02 seconds

Between computers (or virtual machines)

install Puppet using the shell script above.

 

If you run the agent as above, you’ll get this error:

Exiting; no certificate found and waitforcert is disabled

 

on the server, ensure puppet master is running, then

on the client machine, ensure puppet.conf has the server property set to the hostname of the server (or use –server in the command below) and do:

sudo puppet agent –waitforcert 60 –test –debug –no-daemonize

then on the server, on the a different shell than the on running puppet master, do:

<span class="kw2">sudo</span> puppetca <span class="re5">--sign</span> <hostname of the client machine>

 

until the command above is run on the server, the client will output the following message:

notice: Did not receive certificate

Then when the command to issue a certificate is run on the server, the server will output:

notice: Signed certificate request for <client host name>




notice: Removing file Puppet::SSL::CertificateRequest mc-s056627.home at '/etc/puppet/ssl/ca/requests/<client hostname>.pem'

and the client will output:

info: Caching certificate for <client hostname>

Revoking a client’s privilege to connect to the Puppet master

the client certificate’s name is the lowercase hostname

 

To revoke a client’s certificate and thus deny it’s connection attempts, it’s a two  steps process.

First on the server, revoke the certificates:

sudo puppetca --revoke <client host name>

Then on the client, remove the certificates:

sudo rm -rf /etc/puppet/ssl

In some circumstances, you will need to use the following command to completely remove the client certificate from the master:

sudo puppet cert clean <client hostname>

Gotchas

Ruby errors:

If you encounter one of the following errors:

/usr/bin/puppet:3:in `require': no such file to load -- puppet/util/command_line (LoadError)        from /usr/bin/puppet:3/usr/sbin/puppetd:3:in `require': no such file to load -- puppet/application/agent (LoadError)        from /usr/sbin/puppetd:3

it’s probably because you’ve got rvm installed.

 

You can make the problem go away by using system ruby:

rvm use system

I’m not happy about that solution, but I haven’t find a better way so far.

Error about plugins when running the puppet agent:

If you see this error in the agent log:

info: Retrieving plugin




err: /File[/var/lib/puppet/lib]: Could not evaluate: Could not retrieve information from environment production source(s) puppet://wall-e.home/plugins

The puppet master log would correspondingly display this:

info: Could not find filesystem info for file 'plugins' in environment production




info: Could not find file_metadata for 'plugins'

ensure you’ve set pluginsync to false in the puppet.conf: pluginsync = false

Certificates errors:

so far, I solved them by deleting the /etc/puppet/ssl directory on client and master

 

What if problems occur

  • use –debug and –verbose options to puppet commands

  • use –configprint to dump the value of a config property. E.g: puppet apply –configprint modulepath

  • use the notify keyword in your classes to print custom debug information

  • check the documentation

Links

[1] http://trevmex.com/post/850520511/bootstrapping-puppet-on-mac-os-x

 

Language guide: http://docs.puppetlabs.com/guides/language_guide.html

Modules and classes: http://docs.puppetlabs.com/learning/modules1.html

Core Types Cheat sheet: http://docs.puppetlabs.com/puppet_core_types_cheatsheet.pdf

 

Networking in Debian 4.0 (Etch) VMWare Images

Recently I’ve come across a couple of vmware images of Debian Linux 4.0 (Etch) where the network didn’t work. A call to ifconfig didn’t show ‘eth0’ at all.

I’ve tried to compare the configuration difference between a successully running Debian Linux 5.0 (Lenny) and these Etch image without success.

I found the solution by chance on a vmware image vendor’s web site:

rm /etc/udev/rules.d/z25_persistent-net.rules && reboot

Git

I’ve been kind of working on a git tutorial for a while now. I’ve recently realized that there are two different work flows ( git+subversion and github.com) that I use regularly and that trying to describe them both in one tutorial made it quite confusing.

I’m going to remake them in two tutorials instead of one. In the meantime here’s the content of my bash profile with the elements I use to smooth my daily experience of git:

#### start Git #####
alias g='git'
alias gco='git checkout'
alias gma='git checkout master'
alias gst='git status'
alias glo='git log'
alias gca='git commit -a'
alias gsd='git svn dcommit'
alias gcav='git commit -v -a'
alias squash='git merge --squash'
alias gpatch='git format-patch'
alias saw='git branch -D'
alias rollback='git reset --hard git-svn'
alias uncommit='git reset --mixed HEAD'
alias fixlastcommit='git commit --amend'
alias branches='git branch -a'
alias grow='git checkout -b'
alias plant='git svn init'
alias gclone='git clone'
alias hide='git stash'
alias unhide='git stash apply'


export PS1='\w $(git branch &>/dev/null; if [ $? -eq 0 ]; then \
echo "(\[\033[00m\]$(git branch | grep ^*|sed s/\*\ //)) "; fi)\$\[\033[00m\] '

#### end Git #####

Update to the MogileFS Setup Guide

I’ve done minor changes to  the installaion guide for MogileFS following feedback from Craig.

In addition, he noticed that on some Xen VMs, you may encounter the following error:

ERROR: Need to be root to increase max connections

in which case, you will need to update /etc/security/limits.conf with the values:

mogile soft nofile 65535 mogile hard nofile 65535

Testing flash.now With Rspec

I’ve spent a couple of hour trying to test a Rails controller. More specifically one of the action is suppose to display flash.now notice and I want to test that and it works. It took me awhile and some googling to realise that the content of a flash.now is deleted after the action, so that you cannot test it the same way as a normal flash (that last for the duration of the current action and the next one).

I came an across an elegant solution  to this problem as decribed on this blog.

Setting Up High Availability Storage With MogileFS

1. Environment

I used 4 Xen virtual images running ubuntu 8.04. Two will run a tracker and the database, the other two will be the storage nodes.

Lets say the IP addresses will be:

192.168.0.195 192.168.0.196 192.168.0.197 192.168.0.198

2. Initial Setup

Install iptables:

apt-get install iptables

then apply initial setup

iptables -A INPUT -i lo -j ACCEPT iptables -A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT iptables -A INPUT -p tcp --dport ssh -j ACCEPT iptables -A INPUT -j DROP iptables-save -c > /etc/iptables.rules you can save the rule for beyond machine reboot by adding the two lines below to /etc/network/interfaces pre-up iptables-restore < /etc/iptables.rules post-down iptables-save -c > /etc/iptables.rules

install mysql and wget and perldoc

apt-get install mysql-server apt-get install wget apt-get install perl-doc apt-get install libio-aio-perl apt-get install subversion apt-get install build-essential

3. Common steps for installing MogileFS

retrieve the code (ripped of the how to wiki):

cd /usr/local/src mkdir mogilefs-src cd mogilefs-src svn checkout http://code.sixapart.com/svn/mogilefs/trunk

install perl dependencies

cpan Danga::Socket cpan Gearman::Client cpan Gearman::Server cpan Gearman::Client::Async cpan Net::Netmask cpan Perlbal cpan IO::WrapTie

install the servers:

cd mogilefs-src/trunk/server perl Makefile.PL make make test make install

At the moment the test seems to need mysql to be installed and user root without password, so some tests are skipped

you will need to install MogileFS::Client (the tests expect a tracker to run locally on port 7001)

cd mogilefs-src/trunk/api/perl/MogileFS-Client perl Makefile.PL make make test make install

and some admin tools:

cd mogilefs-src/trunk/utils perl Makefile.PL make make test make install

4. Tracker install

create the database:

mysql -uroot -p mysql> CREATE DATABASE mogilefs; mysql> GRANT ALL ON mogilefs.* TO 'mogile'@'%'; mysql> SET PASSWORD FOR 'mogile'@'%' = OLD_PASSWORD( 'sekrit' ); mysql> FLUSH PRIVILEGES; mysql> quit

Create the schema

./mogdbsetup --dbname=mogilefs --dbuser=mogile --dbpassword=sekrit

(admin privilege is required for the initial setup, so if you’re admin user is not root with no password, you will need to specify –dbroopassword and –dbrootuser)

create /etc/mogilefs/mogilefsd.conf:

db_dsn DBI:mysql:mogilefs db_user mogile db_pass ****** conf_port 7001 listener_jobs 5

create a mogile user:

adduser mogile

and starts the tracker under that user:

su - mogile mogilefsd

open a port for the tracker

iptables -A INPUT -p tcp --dport 7001 -j ACCEPT

5. Storage node

On the storage server, create a configuration file at /etc/mogilefs/mogstored.conf with the following:

httplisten=0.0.0.0:7500 mgmtlisten=0.0.0.0:7501 docroot=/var/mogdata

open a port:

iptables -A INPUT -p tcp --dport 7500 -j ACCEPT

iptables -A INPUT -p tcp --dport 7501 -j ACCEPT

register a new storage node:

mogadm --lib=/usr/local/share/perl/5.8.8 --trackers=192.168.0.195:7001 host add mogilestorage --ip=192.168.0.197 --port=7500 --status=alive

it should now appears in the list:

mogadm --lib=/usr/local/share/perl/5.8.8 --trackers=192.168.0.195:7001 host list

Add a device to the storage:

mogadm --lib=/usr/local/share/perl/5.8.8 --trackers=192.168.0.195:7001 device add mogilestorage 1

and create the directory:

mkdir -p /var/mogdata/dev1

make sure /var/mogdata/* is owned by mogile:mogile

chown -R mogile:mogile /var/mogdata/*

6. Starting the storage server

as root: mogstored --daemon

7. Starting the tracker

su - mogile mogilefsd -c /etc/mogilefs/mogilefsd.conf --daemon exit

8. Testing

check that mogilefs components are online:

mogadm --lib=/usr/local/share/perl/5.8.8 --trackers=192.168.0.195:7001 check

Quick sanity check of the storage daemon:

~/Projects/mogilefs $ telnet 192.168.0.197 7500 Trying 192.168.0.197... Connected to 192.168.0.197. Escape character is '^]'. PUT /dev1/test HTTP/1.0 Content-length: 4 \n test HTTP/1.0 200 OK Content-Type: text/html Content-Length: 18 Server: Perlbal Connection: close 200 - OK Connection closed by foreign host.

create a domain

mogadm --lib=/usr/local/share/perl/5.8.8 --trackers=192.168.0.195:7001 domain add mydomain

and a class

mogadm --lib=/usr/local/share/perl/5.8.8 --trackers=192.168.0.195:7001 class add mydomain images

Quick sanity check of the tracker:

root@bbc-01:~# mogtool --trackers=127.0.0.1:7001 --domain=mydomain --class=images inject osname osname

on the store node, check /var/mogdata/dev1/0/000/000 for a file named xxxxxxxxxx.fid . If the file exists it’s all good.

9. setting up the second pair

Replay instructions 1 to 8, then:

when you’ve got the second storage set up, you will need to register the second storage and its device to all trackers:

mkdir -p /var/mogdata/dev2 mogadm --lib=/usr/local/share/perl/5.8.8 --trackers=192.168.0.195:7001 host add mogilestorage2 --ip=192.168.0.198 --port=7500 --status=alive mogadm --lib=/usr/local/share/perl/5.8.8 --trackers=192.168.0.195:7001 device add mogilestorage2 2 mogadm --lib=/usr/local/share/perl/5.8.8 --trackers=192.168.0.196:7001 host add mogilestorage2 --ip=192.168.0.198 --port=7500 --status=alive mogadm --lib=/usr/local/share/perl/5.8.8 --trackers=192.168.0.196:7001 device add mogilestorage2 2

you will also need to register the first storage and its device to the second tracker:

mogadm --lib=/usr/local/share/perl/5.8.8 --trackers=192.168.0.196:7001 host add mogilestorage --ip=192.168.0.197 --port=7500 --status=alive mogadm --lib=/usr/local/share/perl/5.8.8 --trackers=192.168.0.196:7001 device add mogilestorage 1

sanity check the installation:

mogadm --lib=/usr/local/share/perl/5.8.8 --trackers=192.168.0.195:7001,192.168.0.196:7001 check

Checking trackers… 192.168.0.195:7001 … OK 192.168.0.196:7001 … OK

Checking hosts… [ 1] mogilestorage2 … OK [ 2] mogilestorage … OK

Checking devices… host device size(G) used(G) free(G) use% ob state I/O%


[ 1] dev2 9.921 0.757 9.164 7.63% writeable 0.0 [ 2] dev1 9.921 0.600 9.321 6.05% writeable 0.0


total: 19.842 1.357 18.485 6.84%

And voila.

Beach Close-ups (Cc)

[Beach close-ups (cc)](http://www.flickr.com/photos/dunstan/2398525947/)


Originally uploaded by [Dunstan Orchard](http://www.flickr.com/people/dunstan/)

Flickr has today announced support for video media.

I’m impressed on how the integration is done, it’s the same experience as for photo plus the added value of what they call “long photo”.

Very elegant indeed.