Today I stumbled over http://scrapy.org/ while searching for an OpenSource website crawler. Its an interesting crawling and scraping framework for Python. It looks very convenient and easy to use. The most interesting feature seems to be the possibility to select website elements (f.e. hyperlinks) via CSS-selectors. In any case I’ll give it a try.
Recently I brought a mp3 player that wasn’t able to shuffle the track through all folders. That’s why I had to bring all tracks to the root directory of the player (flatten directory structure). I did it with the following command, that flattens the whole structure by searching all files in the structure and copying […]
Sometimes I had problems to connect to a server via public key authentication. In most cases I could solve them by setting up the right permissions for the file and path of the authorized_keys file. This can be done as shown below:
|
1 2 3 4 |
$> chmod 700 $HOME/.ssh $> chmod 600 $HOME/.ssh/authorized_keys $> chmod go-w $HOME $HOME/.ssh $> chown `whoami` $HOME/.ssh/authorized_keys |
To get information about the currently connected user and more MySQL status information, you can call the following command if you are already logged in:
|
1 |
mysql> STATUS; |
This will give you an output like that:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 |
-------------- mysql Ver 14.14 Distrib 5.1.73, for redhat-linux-gnu (x86_64) using readline 5.1 Connection id: 1044395 Current database: Current user: niceguy@localhost SSL: Not in use Current pager: stdout Using outfile: '' Using delimiter: ; Server version: 5.1.73-log Source distribution Protocol version: 10 Connection: Localhost via UNIX socket Server characterset: latin1 Db characterset: latin1 Client characterset: latin1 Conn. characterset: latin1 UNIX socket: /var/lib/mysql/mysql.sock Uptime: 2 days 6 hours 26 min 44 sec Threads: 8 Questions: 399680642 Slow queries: 20292 Opens: 9697 Flush tables: 1 Open tables: 4096 Queries per second avg: 2039.145 -------------- |
As you can see, you got a lot more information like operation system data, MysSQL version or the number of slow […]
This simple bash script is able to backup all given files and folders to a backup destination. I use it to backup all my Raspberry Pis, Banana Pis and also my notebook to my Synology NAS.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 |
#!/bin/bash # Lockfilecheck for race condition protection exec 9>.backupLockFile if ! flock -n 9 ; then echo "Another instance is already running dude!"; exit 1; fi #echo "1: $1"; #echo "2: $2"; #echo "3: $3"; #echo "4: $4"; DATE=`date +%y%m%d`; SOURCES="/*"; if [ "$1" == "help" ] then echo "Help"; echo "- USAGE: bash backup_to_synology.sh [DIRTOBACKUP] [EXCLUDEFILEPATH] [BACKUPDESTINATION] [disable-dry-run]"; echo "- OR: bash backup_to_synology.sh null null [BACKUPDESTINATION] [disable-dry-run]"; echo "- OR: bash backup_to_synology.sh [DIRTOBACKUP] null [BACKUPDESTINATION] [disable-dry-run]"; echo "- OR: bash backup_to_synology.sh [DIRTOBACKUP] null [BACKUPDESTINATION] [disable-dry-run] [logfile]"; exit 0; elif [ -n "$1" -a "$1" != "null" ] then SOURCES="$1"; fi EXCLUDEFILE=""; if [ -n "$2" -a "$2" != "null" ] then EXCLUDEFILE="--exclude-from=$2"; fi # Define remote target TARGET=""; if [ -n "$3" -a "$3" != "null" ] then TARGET="$3"; else echo "You have to give me a destination"; exit 1; fi DRYRUN="--dry-run"; if [ -n "$4" -a "$4" == "disable-dry-run" ] then DRYRUN=""; fi # rsync base params # --delete - delete destination files that dosn't exist on the source # --exclude-from - file / folders to ignore can be # # use -–dry-run to simulate backup -> this is the default mode unless disable-dry-run is set PARAMS="-azvrpuE --delete"; # Path to logfile LOGFILE="rsync_log.$DATE.log"; if [ -n "$5" -a "$5" != "null" ] then LOGFILE="$5"; fi # determine the path to "rsync" RSYNC=`which rsync`; # run backup command $RSYNC $PARAMS $DRYRUN $EXCLUDEFILE $SOURCES $TARGET > $LOGFILE; exit 0; |
Example call:
|
1 |
$> bash backup_to_synology.sh "/data/snots/*" exclude.txt backup@ds.intern.ask-sheldon.com:/volume1/NetBackup/thinkpad disable-dry-run logfile.log |
If you want to run it as a crontab, you can use key based authentication (see […]
To crate a UTF-8 database:
|
1 |
CREATE DATABASE sns_typo3 DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci; |
To reinstall Magtento with the installation wizard you have to: delete app/etc/local.xml clear cache
|
1 |
$> rm -rf /path/to/mage/var/cache/* |
reload your shop frontend
To change the permissions of all files and / or folders insight the current folder recursively you can run one of the following command:
|
1 2 3 |
$> sudo find . -type d -exec chmod 644 {} \; #all folders get 644 $> sudo find . -exec chmod g+r {} \; #all files and folders get group read rights $> sudo find . -type f -exec chmod 755 {} \; #all files get 755 |
All options can be found on http://unixhelp.ed.ac.uk/CGI/man-cgi?find.
To change the ownership of all files and folders with a certain group and / or user insight the current folder recursively you can run the following command:
|
1 |
$> sudo find . -group OLDGROUP -user OLDUSER -exec chown NEWUSER:NEWGROUP {} \; |
This way you can set the owning group and / or user.
It’s no big deal to comment out a line in YAML. Just put a sharp (#) as the first character of that line.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 |
fileSync: typo3_user_upload: #rsync_params: '--delete -avhHxS' #source: 'shared/typomage/fileadmin/user_upload' #target: 'shared/typomage/fileadmin/user_upload' typo3_redaktion: rsync_params: '--delete -avhHxS' source: 'shared/typomage/fileadmin/redaktion' target: 'shared/typomage/fileadmin/redaktion' # magento_media: # rsync_params: '--delete -avhHxS' # source: 'shared/typomage/media/catalog/product' # target: 'shared/typomage/media/catalog/product' |
As you can see the lines 3-5 and 10-13 will be interpreted as comments. Unfortunately there is no block comment syntax in YAML (at least I didn’t find any 😉 !).
