unixTools

  • Unix tools introduced. Today: readonly

    In bash you can set a variable to ‚readonly‘. This can be very useful to prevent accidental overwrites.

    readonly INSTALLDIR="/opt/goodSoftware"
    Weiterlesen: Unix tools introduced. Today: readonly

  • Unix tools introduced. Today: tmux

    With tmux you can share a shell session between different users.

    It provides an easy way to work together in a bash shell from different locations.

    It is as easy as:

    User 1

    $ ssh localhost
$ tmux

    User 2

    $ ssh localhost
$ tmux attach

    Now both users are in a shared terminal environment and can input/output to the same terminal.

    More useful example can be found here:

    https://wiki.ubuntuusers.de/tmux/

    Weiterlesen: Unix tools introduced. Today: tmux

  • Unix tools introduced. Today: pushd and popd

    With pushd  you can bookmark directory locations in a stack datastructure. With popd you can navigate back to them just if you were using a back button in a graphical ui. This can be particular helpful when managing navigation tasks within scripts.

    https://opensource.com/article/19/8/navigating-bash-shell-pushd-popd

    Weiterlesen: Unix tools introduced. Today: pushd and popd

  • Unix tools introduced. Today: cat

    cat is a well known command to concatenate the content of multiple files.  Example: cat file1 file2 file3

    But there are other use cases. cat offers a nice way to print out multi line strings.  It is even possible to include variables into the string, which feels a little bit like using a templating language.

    Example:

    NAME=ADMIN@COMPANY.COM;
cat <<EOF
Hello $LOGNAME,
please be aware. This system will be under maintenance soon.
Have a good day.
Sincerely
$NAME
EOF

    For more info on the <<EOF visit this SO-Thread

     

     

    Weiterlesen: Unix tools introduced. Today: cat

  • Unix tools introduced. Today: rsync

    rsync is a very cool tool that can be used to copy files between hosts or between directories on the same host. Like the term ’sync‘ suggests the copy process can be controlled into great detail to modulate rsync’s behavior.  Take a look at the available options under: https://linux.die.net/man/1/rsync

    This is my list of cool options.  I start with the most basic usage. The following command can be used to copy, and later on sync two directories.

    rsync -avn /source/dir /target/dir

    The command ‚archives‘ file attributes (-a) and displays some status info (-v).

    In the given form, the command only does a dry-run (-n). To execute the command remove the -n.

    The command uses the short form of --archive (-a) which translates to (-rlptgoD).

    • -r – recursive copy
    • -l – copy symlinks as symlinks
    • -p – set target permissions to be the same as the source
    • -t – set target mtime to be the same as the source. Use this to support fast incremental updates based on mtime.
    • -g – set target group to be the same as the source
    • -o – set target owner to be the same as the source
    • -D – if remote user is superuser this recreates devices and other special files.

    More cool options

    Move

    --remove-source-files This will remove copied files from source.

    Update

    --update This forces rsync to skip any files which exist on the destination and have a modified time that is newer than the source file.

    Delete

    --delete Delete files on target that do not exist in source tree.

    Backup

    --backup Make a backup of modified or removed files on target.

    --backup-dir=date +%Y.%m.%d Specify a backup dir on target.

    What to copy?

    --min-size=1 Do not copy empty files. This can be particularly interesting if you have corrupted files in the source.

    --max-size=100K Copy only small files. Can be used to handle small and large files differently.

    --existing Only override files that already exist on the target. Do not create new files on target.

    --ignore-existing Only copy files that do not exist on target.

    --exclude-from Define excludes in a file.

    Scheduling, Bandwidth and Performance

    --time-limit Ends rsync after a certain time limit.

    --stop-at=y-m-dTh:m Ends rsync at a specific time.

    --partial Allows partial copies in case of interruptions.

    --bwlimit=100 Limits bandwidth Specify KBytes/second. Good option if transfer of large files is required.

    Output

    • -h output numbers in a human-readable format.
    • --progress display progress.
    • -i log change info.
    • --log-file= define a log file.
    • --quiet no output.
    •  -v Output status info. You can add more ‚v‘.
    • Forgot to log any progress info? Use the following command to see what rsync is about to do. 
       ls -l /proc/$(pidof rsync)/fd/*

     

    Weiterlesen: Unix tools introduced. Today: rsync

  • Unix tools introduced. Today: offlineimap

    Offlineimap is a tool to backup or sync IMAP accounts. Backing up an IMAP account to a local folder is easy.

    Create a file `.offlineimaprc` in your home directory like described here:  http://www.offlineimap.org/doc/quick_start.html.

    Now you can run `offlineimap` from your command line. Type in your password and the download will start.

    Weiterlesen: Unix tools introduced. Today: offlineimap

  • Unix tools introduced. Today: pgrep

    pgrep looks through the currently running processes and lists the process IDs. Use

    pgrep -f 'process_name'

    to find the process ID of a process.

    Weiterlesen: Unix tools introduced. Today: pgrep

  • Unix tools introduced. Today: Readonly SFTP

    To create a user (sftp) with readonly  access via sftp to a single directory (/var/sftp_readonly), perform the following steps:

    sudo su #become root

useradd sftp #create new user

passwd sftp #set a password

groupadd sftp_readonly #create a group

mkdir /var/sftp_readonly #create a directory

usermod -G sftp_readonly sftp # add user to group

chmod 755 /var/sftp_readonly/ #allow others to read

cp /etc/ssh/sshd_config ~/sshd_config.bck #backup your ssh config

editor  /etc/ssh/sshd_config # edit your ssh_config

    Add the following lines to the bottom of /etc/ssh/sshd_config

    Match Group sftp_readonly
  X11Forwarding no
  AllowTcpForwarding no
  ChrootDirectory /var/sftp_readonly/
  ForceCommand internal-sftp

    Also make sure that the following line is present

    Subsystem sftp internal-sftp

    Reload your ssh service

    sudo service ssh reload

     

     

    Weiterlesen: Unix tools introduced. Today: Readonly SFTP

  • Unix tools introduced. Today: FHS

    The Filesystem Hierarchy Standard (FHS) defines a standard layout to organize various kinds of application and OS related data in a predictable and common way [1].

    A basic knowledge of the FHS will help you to find application or OS related data more easily. If you are a developer, it also provides a good orientation for organizing your own applications in a maintainable way, e.g. as ubuntu package.

    /bin – essential user commands

    /boot – OS boot loader

    /dev – devices (everything is a file principle)

    /etc – system configuration

    /home – user data

    /lib – essentail shared libraries

    /media – mount point for removable media

    /mnt – mount point for temporarily mounted filesystems

    /opt – add-on applications

    /root – home of root

    /run – run time variable data

    /sbin – system binaries

    /srv – data for services provided by the system

    /tmp – temporary data

    /proc – is a virtual filesystem

    /usr – secondary hierarchy

    bin – Most user commands
    lib – Libraries
    local – Local hierarchy (empty after main installation)
    sbin – Non-vital system binaries
    share – Architecture-independent data

    /var – variable data

    cache  – Application cache data
    lib  – Variable state information
    local  – Variable data for /usr/local
    lock –  Lock files
    log – Log files and directories
    opt – Variable data for /opt
    run – Data relevant to running processes
    spool – Application spool data
    tmp  -Temporary files preserved between system reboots

    Find more

    What about – /init.d ?

    What does the .d stand for in directory names?

    FHS in Debian

     

    Weiterlesen: Unix tools introduced. Today: FHS

  • Unix tools introduced. Today: date

    The unix tool date can be used to print the current date. It also can be used to calculate time spans or to reformat time strings.

    Examples:
    1. Print the current date in a conventional form

    date +"%d. %B %Y"

    2. Use certain locale. Find supported list in /usr/share/i18n/SUPPORTED

    LANG=de_DE.UTF-8 date +"%d. %B %Y"

    3. Print an arbitrary (but valid) timestring in a conventional form

    date -d 2003-11-22 +"%d. %B %Y"

    4. Print the current date as seconds since 1970 (unix epoche)

    date +%s

    5. Calculate the difference of two dates in days. Explanation can be found here.

    A="2002-10-20"
B="2003-11-22"
echo $(( ($(date -d $B +%s) - $(date -d $A +%s)) / 86400 )) days

    6. Print seconds after 1970 (unix epoche) in a conventional form.

    TZ=UTC LANG=en_EN.UTF-8  date -d@0 +"%H:%M:%S %d. %B %Y"

    Note: if TZ=UTC is left out, date will add or subtract an offset in accordance to systems timezone.

    7. Find timezone offset

    OFF_1=$(date -d '1 Jan' +%z)
OFF_7=$(date -d '1 Jul' +%z)
echo $OFF_1 $OFF_7

     

    Weiterlesen: Unix tools introduced. Today: date