Nice Python website crawler framework

Today I stumbled over http://scrapy.org/ while searching for an OpenSource website crawler. Its an interesting crawling and scraping framework for Python. It looks very convenient and easy to use. The most interesting feature seems to be the possibility to select website elements (f.e. hyperlinks) via CSS-selectors. In any case I’ll give it a try.

Flatten directory structure

Recently I brought a mp3 player that wasn’t able to shuffle the track through all folders. That’s why I had to bring all tracks to the root directory of the player (flatten directory structure). I did it with the following command, that flattens the whole structure by searching all files in the structure and copying […]

Resolve problems with authorized_keys permissions

Sometimes I had problems to connect to a server via public key authentication. In most cases I could solve them by setting up the right permissions for the file and path of the authorized_keys file. This can be done as shown below: $> chmod 700 $HOME/.ssh $> chmod 600 $HOME/.ssh/authorized_keys $> chmod go-w $HOME $HOME/.ssh […]

Get current user, mysql version and other status information

To get information about the currently connected user and more MySQL status information, you can call the following command if you are already logged in: This will give you an output like that: ————– mysql Ver 14.14 Distrib 5.1.73, for redhat-linux-gnu (x86_64) using readline 5.1 Connection id: 1044395 Current database: Current user: niceguy@localhost SSL: Not in […]

Simple rsync based backup script

This simple bash script is able to backup all given files and folders to a backup destination. I use it to backup all my Raspberry Pis, Banana Pis and also my notebook to my Synology NAS. #!/bin/bash # Lockfilecheck for race condition protection exec 9>.backupLockFile if ! flock -n 9 ; then echo “Another instance is […]

Comments in YAML

It’s no big deal to comment out a line in YAML. Just put a sharp (#) as the first character of that line. fileSync: typo3_user_upload: #rsync_params: ‘–delete -avhHxS’ #source: ‘shared/typomage/fileadmin/user_upload’ #target: ‘shared/typomage/fileadmin/user_upload’ typo3_redaktion: rsync_params: ‘–delete -avhHxS’ source: ‘shared/typomage/fileadmin/redaktion’ target: ‘shared/typomage/fileadmin/redaktion’ # magento_media: # rsync_params: ‘–delete -avhHxS’ # source: ‘shared/typomage/media/catalog/product’ # target: ‘shared/typomage/media/catalog/product’ As you can see […]