Nice Python website crawler framework

Today I stumbled over http://scrapy.org/ while searching for an OpenSource website crawler. Its an interesting crawling and scraping framework for Python. It looks very convenient and easy to use. The most interesting feature seems to be the possibility to select website elements (f.e. hyperlinks) via CSS-selectors. In any case I’ll give it a try.

Flatten directory structure

Recently I brought a mp3 player that wasn’t able to shuffle the track through all folders. That’s why I had to bring all tracks to the root directory of the player (flatten directory structure). I did it with the following command, that flattens the whole structure by searching all files in the structure and copying […]

Resolve problems with authorized_keys permissions

Sometimes I had problems to connect to a server via public key authentication. In most cases I could solve them by setting up the right permissions for the file and path of the authorized_keys file. This can be done as shown below:

Get current user, mysql version and other status information

To get information about the currently connected user and more MySQL status information, you can call the following command if you are already logged in:

This will give you an output like that:

As you can see, you got a lot more information like operation system data, MysSQL version or the number of slow […]

Simple rsync based backup script

This simple bash script is able to backup all given files and folders to a backup destination. I use it to backup all my Raspberry Pis, Banana Pis and also my notebook to my Synology NAS.

Example call:

If you want to run it as a crontab, you can use key based authentication (see […]

Recursively set permissions

To change the permissions of all files and / or folders insight the current folder recursively you can run one of  the following command:

All options can be found on http://unixhelp.ed.ac.uk/CGI/man-cgi?find.

Recursively assign ownership to files and folders

To change the ownership of all files and folders with a certain group and / or user insight the current folder recursively you can run the following command:

This way you can set the owning group and / or user.

Comments in YAML

It’s no big deal to comment out a line in YAML. Just put a sharp (#) as the first character of that line.

As you can see the lines 3-5 and 10-13 will be interpreted  as comments. Unfortunately there is no block comment syntax in YAML (at least I didn’t find any 😉 !).