peterlog

the web log of peter sabaini

  • Home
  • About
    • 0
      12 May 2012

      The Cubox Has Landed

      • Edit
      • Delete
      • Tags
      • Autopost

      Finally (and thanks to some nice people in Graz who resent a misaddressed package -- thanks guys!) my Cubox has arrived! It really is a cute little box:

      Bfhdfeij


      The Cubox is a nice little ARM-based computer, capable of running Linux and decoding HD content. It sports HDMI and SPDIF outputs, and boots from an internal SD-card. It has minimal power requirements (3 Watts when streaming, <1W in standby) and is real quiet, as in, you don't hear it at all. It has no internal storage but can mount USB drives, and, even more importantly, networked drives.

      I intend to replace my ageing media center PC with this. It did it's job, but was relatively noisy and drew a lot more power (ie. I always turned it off when done and had to accomodate the boot time).

      Installing XBMC was quite straightforward thanks to the Wiki. I still need to set up the SPDIF stuff, and also need a TOSLink cable (I'm usually not into high end audio but I hope to combat some hum I get when using regular cabling to the amp).

      Bootup:


      [    0.000000] Linux version 2.6.32.9-dove-5.4.2 (rabeeh@kossay-desktop) (gcc version 4.3.2 (sdk3.2rc1-ct-ng-1.4.1) ) #46 PREEMPT Sat Jan 7 12:23:38 IST 2012
      [    0.000000] CPU: Marvell PJ4 v7 Processor-wmmx [560f5815] revision 5 (ARMv7), cr=10c53c7f
      [    0.000000] CPU: VIPT nonaliasing data cache, VIPT nonaliasing instruction cache
      [    0.000000] Machine: SolidRun CuBox Platform
      [    0.000000] Marvell Dove DRAM parameters found (version = 0x09080000)
      [    0.000000] Marvell Dove DVS parameters found (version = 0x09080000)
      [    0.000000] Memory policy: ECC disabled, Data cache writeback
      [    0.000000] On node 0 totalpages: 163840
      [    0.000000] free_area_init_node: node 0, pgdat c0756e30, node_mem_map c07ff000
      [    0.000000]   Normal zone: 1200 pages used for memmap
      [    0.000000]   Normal zone: 0 pages reserved
      [    0.000000]   Normal zone: 152400 pages, LIFO batch:31

      • views
      • Tweet
    • 0
      31 Jan 2012

      Easy Bash Completion

      • Edit
      • Delete
      • Tags
      • Autopost

      I’m a lazy/impatient typist and therefore a big fan of shell completion. There’s a lot of good completion support for common programs, but what if I want to have shell completion for a custom program or shell function? A quick and easy way is to bind one of the pre-defined completion functions, as I learned today.

      Eg. I have a shell function “sshr” to log me into a remote host as root:

      sshr() { /usr/bin/ssh "root@$@" ;}

      To have the shell use the same expansion rules as regular ssh, bind it like this:

      complete -F _ssh sshr

      To get a list of predefined completion rules, run “complete -p”

      This and more from

      • Debian Admin post, pt1
      • Debian Admin post, pt2
      • views
      • Tweet
    • 0
      22 Jan 2012

      Puppet & Ruby

      • Edit
      • Delete
      • Tags
      • Autopost

      So I spent the last week in Amsterdam for training. Amsterdam is a great city, so it's unfortunate that I didn't have all that much time for exploration. Luckily I at least managed a small trip to the Rijksmuseum; while most of the museum is closed for restoration, they have a small selection with truly great pieces on display.

      Ijfafjjd

      Training was also very enjoyable however. Topic of the training was the IT automation toolkit Puppet, and, since Puppet is written in Ruby, also a little of that was covered. Mr Haugen from Puppetlabs did an excellent job as an instructor; with just the right mix of theory presentation and exercises (and in between answering lots of questions).

      I have a bit of a love/hate relationship with Puppet. I've been using Puppet before; it sure is a very useful tool but on the other hand... It does have some warts.  

      For one the terminology of the DSL. For someone who has done OO programming the word "class" carries very specifc meaning. Puppet also has classes, but the are almost entirely unlike those from OOP. For example, all Puppet classes are singletons and may not be instantiated more than once. Also, there is some inheritance but overriding parameters is not supported; so the advice is to avoid inheritance and use includes. Include is not at all like someone with a C (or Erlang) background might expect it to be; it's not a simple literal file include but actually a kind of import + instantiate function. And so on.

      Jbiebdfj

      On the other hand, I much like the declarative model and the fact that it is very open and extensible. For instance, configuration parameters need not be held in static files but may also reside in an external repository; all that is needed is a glue script to fetch said parameters.

      Also Puppets approach is refreshingly pragmatic; it's all about getting stuff done. One can easily see that there are actual sysadmins using this stuff in production. It has decent support for testing and dry runs, for example -- good to have if you're not that confident about a system change you're going to roll out to a few thousand hosts. 

      It actually reminded me a little bit of the earlier days of Zope2 -- lots of magic and little documenation; for someone who knows how to push the right buttons a lot can be done with little effort but to get there you have a not so friendly learning curve.

      Puppet is written in Ruby, and for many types of extensions you have to get down to the Ruby level.

      Bcejeddf

      I haven't done much with Ruby before, so I'd like to take up the opportuniy write up some thoughts about the language and platform. My reference here is mainly Python, a) because these languages are in a pretty similar niche and b) Python is what I know best.

      For this Python programmer, Ruby tends to feel a little "Perlish" from time to time. In cases where Python might choose to focus on "one right way" to do something, Ruby would rather provide several ways (in different flavors) to accomplish the same thing. This sometimes can be quite useful, sometimes rather confusing. For me, this makes Python more intuitively useful, but admittedly sometimes more verbose. 

      One of the Perlish features of Ruby that I quite like are built-in regular expressions. For Sysadmins who constantly have to wrangle output of other tools to their own end regular expressions are a must have.  While Python of course has an excellent library for RE's it is still not the same level of integration as a native datatype with accompanying operators etc. On the other hand Ruby has Perls habit of setting tons of special global variables for RE matches -- not trying to be dogmatic here but can we maybe just try to avoid automatic global vars? Please? People get hurt by that stuff! This, to be fair, is entirely possible because Ruby _also_ provides other ways of accessing these values, without resorting to globals.

      One of the Perlish things in Ruby that I wish they just had left out is the postfix conditional, ie. one can write "if x then foobar()" but also "foobar() if x". Larry Wall has stressed how that helps make Perl like writing English but in all honesty I'd like to have my software a little more regular than the English language with its abundance of corner cases and special rules for all kinds of occasions.

      Speaking of conditionals -- many things which would be statements in other languages are actually expressions in Ruby, ie. they have a value; also including if-conditions. Eg. this function would add 2 to numbers smaller than 23 (since functions return the value of their last expression):

             def foo(a)
                 if a < 23
                    a+2
                 else
                   a
                 end
             end

      I find having more expressions lets one focus more on the data than on the procedure of how to operate on it. Yay for expressions! 

      Ruby has a very easy way to parametrize methods with anonymous blocks of code. While this allows for some very higher-order / functional coding style (which I appreciate) I had the impression that feature tends to get somewhat overused (well at least from what I've seen in Puppet). Sometimes it felt as if people we're actively trying to avoid writing regular methods and instead resorting to fiddling with blobs of anonymous code to achieve the same thing. Still, good to have for functional programming.

      Ruby is more consistent in its OO compared to Python, eg. while Python provides a built-in "length()" function, in Ruby only values which actually have a length (ie. Arrays) provide a ".length()" method. Makes a lot more sense, in my opinion.

      By the way, you may call methods with or without parentheses in Ruby. Personally I prefer parentheses for calls, I find code easier to read that way, but that might also just be a matter of practice.

      As to concurrency, Ruby unfortunately has similar problems as Python. The only directly supported concurrency model is a threading one, and like Python Ruby has to serialize access to data through a global interpreter lock. This usually results in Ruby code not scaling up very well on CPU-bound tasks. On the other hand, if you are doing CPU-intensive tasks you might want to consider a C extension anyway; these are luckily quite easy to embed (there even is a library that lets you inline C code into Ruby).

      Iijgcbbj

      Moving away from the core language on to the runtime environment -- there's a lot of good and also some bad there. 

      First to the bad: the standard library is, at least compared to the Python equivalent, a pretty rough place. Many modules are sparsely documented or not at all, and for some of the modules I kinda
      wonder if anyone uses them.

      The good, of course, is the gem system, to the point that I'm thinking maybe gems are the reason the stdlib seems to be lacking in maintenance.

      Gems is Ruby's packaging format and application, and it handles everything a packaging system should -- installation, dependency management, querying, etc. Pythons packaging system(s) are, in comparison, a bit of a mess.

      There are a lot of gems available. Rubygems.org boasts a total of 33160 packages available, and >444 million downloads. While nothing can be infered about the quality and/or usefulness of the provided libraries (and while still below CPANs legendary scale), these are nevertheless some impressive numbers.

      All in all, I'm looking forward to doing more Puppet and Ruby. There's lots to discover!

      • views
      • Tweet
    • 0
      27 Apr 2011

      Be sexy, don't scale

      • Edit
      • Delete
      • Tags
      • Autopost

      Mr Berkus delivers valuable advice on how to be sexy as a developer by not scaling.

      (via HighScalability)

       

      • views
      • Tweet
    • 0
      22 Mar 2011

      Handy I/O Monitoring

      • Edit
      • Delete
      • Tags
      • Autopost
      Screenshot

      Nice little tool someone mentioned at work today: iotop. Iotop shows disk read/write and swapin per process. Very handy if you suspect I/O problems but need to pinpoint the application thats hogging bandwidth! While RAM and CPU get cheaper and cheaper, good I/O subsystems have their price -- and are often heavily taxed in data-intensive applications.

      By default iotop refreshes every second, which is a bit too jittery for my tastes. Use the -d secs switch to set a custom refresh interval. The -o  switch tells iotop to only display those processes actually doing I/O; hitting 'o' while running iotop has the same effect.

      If you want to log the output of iotop or use it in a script, iotop supports '-b' for "batch mode" much in the same way as top does.

      Many Linux distributions package iotop, ie. its just an "apt-get install iotop" or "yum install iotop" away.


      • views
      • Tweet
    • 2
      3 Dec 2010

      Zeroconf, .local, Avahi

      • Edit
      • Delete
      • Tags
      • Autopost

      Was ich heute gelernt habe: Zeroconf verwendet die DNS – Toplevel Domain .local, und Anfragen an Namen aus dieser Domain werden nicht mehr von den regulaeren Nameservern behandelt. Das ist jedenfalls das Verhalten das sich mit Avahi unter Ubuntu Maverick zeigt.

      Schlecht wenn man, wie bei uns, die .local TLD fuer firmen-interne Namen verwendet. Die kann man dann nicht mehr aufloesen…

      Ob die Verwendung von erfundenen TLDs eine gute Idee ist, sei dahingestellt. Gut waere es natuerlich, eine offiziell empfohlene TLD fuer interne Verwendung zu haben, analog zu privaten IP-Adressen nach RFC 1918, aber so eine Empfehlung existiert leider nicht. Die Verwendung von .local als TLD fuer firmen-interne Namen war zumindest lang die Empfehlung von Microsoft, und grade auch deshalb finde ich die Wahl von .local durch die Zeroconf-Entwickler eher bedauerlich (war der Name clash beabsichtigt?).

      Auf der Avahi-Seite finden sich einige mehr oder weniger befriedigende Workarounds. Ich habe mal die verwendete TLD auf .alocal umgestellt.

      • views
      • Tweet
    • 2
      12 Jul 2010

      Filesystem Brutality

      • Edit
      • Delete
      • Tags
      • Autopost

      Or, how to scrape data off a broken harddrive with unnecessary cruelty

      The other day a friend of mine sent a terabyte harddrive by which, due to events unknown, had ceased to perform its duty in a Linux-based NAS.

      Indeed, it seemed as if the disk in question 1) indeed contained an ext3 filesystem, and 2) had lost its partition table, and as no information about the partitioning was available, I started off to try the usual tools to guess a layout. I first tried GPart , then Testdisk, both without much success.

      I proceeded to try and find a partition table manually; as there are usually backup copies of this table present. Manual search, however, turned out to be quite tedious, so I searched for a way to automate this.

      I came up with this little Python thing:

      from subprocess import *
      
      def losetup(offs):
          cmd = ['/sbin/losetup', \
                 '-o', str(offs), \
                 '/dev/loop0', '/dev/sdb']
          call(cmd)
      
      def fsck(n):
          print "\nTrying byte %s" % n
          cmd = ['/sbin/fsck.ext2', '-n', '/dev/loop0']
          p = Popen(cmd, stdout=PIPE, stderr=PIPE)
          out, err = p.communicate()
          if p.returncode != 8:
              print out
              print err
      
      def delo():
          cmd = ['/sbin/losetup', '-d', '/dev/loop0']
          call(cmd)
      
      for n in xrange(0, 4096):
          losetup(n)
          fsck(n)
          delo()

      It really is a bit brutal. It iterates through the first 4k bytes of the harddrive (I made a backup image beforehand) and tries to perform a fsck (without actually changing anything). I basically let fsck do the test if a partition table can be found (or a backup). And, much to my surprise, fsck actually found a backup partition table using an offset of 512, so all I had to do was losetup once more, and this time perform the fsck for real. Almost all of the data could be recovered! Yay losetup and Python!

      • views
      • Tweet
    • Search

    • Tags

      • programming
      • linux
      • python
      • switzerland
      • photo
      • posterous
      • zug
      • database
      • android
      • erlang
      • kitteh
      • music
      • tool
      • couchdb
      • electronica
      • teaching
      • work
      • JavaScript
      • NoScript
      • animation
      • aquarium
      • austria
      • biking
      • bug
      • comic
      • conveniencetrumpssecurity
      • dns
      • ehealth
      • elga
      • emacs
      • emkuu
      • fun
      • gaming
      • gesundheitspolitik
      • golang
      • graphic novel
      • html5
      • industrial
      • java
      • latex
      • mail
      • messaging
      • microformats
      • mysql
      • oracle
      • perl
      • politics
      • postgresql
      • puppet
      • rant
      • restructuredtext
      • ruby
      • science
      • security
      • skype
      • styria
      • twisted
      • vintage
      • younggods
      • zeroconf
      • zope
    • Archive

      • 2012 (13)
        • May (2)
        • April (1)
        • March (2)
        • February (1)
        • January (7)
      • 2011 (29)
        • December (5)
        • September (4)
        • August (3)
        • July (1)
        • June (1)
        • April (4)
        • March (5)
        • February (1)
        • January (5)
      • 2010 (35)
        • December (6)
        • November (4)
        • October (3)
        • September (5)
        • August (5)
        • July (11)
        • June (1)
    • Obox Design
  • peterlog

    software, sysadmin, python, zope, erlang, linux, teh interwebs, cats, austria, switzerland

    18898 Views
  • Get Updates

    Follow this Space »
    You're following this Space (Edit)
    You're a contributor here (Edit)
    This is your Space (Edit)
    Follow by email »
    Get the latest updates in your email box automatically.
    Loading...
    Subscribe via RSS
    TwitterFacebookBuzzLinkedIn