Posts

Posted on

Leaving Amazon

https://www.tbray.org/ongoing/When/202x/2020/04/29/Leaving-Amazon

Amazon is exceptionally well-managed and has demonstrated great skill at spotting opportunities and building repeatable processes for exploiting them. It has a corresponding lack of vision about the human costs of the relentless growth and accumulation of wealth and power. If we don’t like certain things Amazon is doing, we need to put legal guardrails in place to stop those things. We don’t need to invent anything new; a combination of antitrust and living-wage and worker-empowerment legislation, rigorously enforced, offers a clear path forward.

Don’t say it can’t be done, because France is doing it.

Posted on

Gesundheitsakte per Injektion

MIT researchers have now developed a novel way to record a patient’s vaccination history: storing the data in a pattern of dye, invisible to the naked eye, that is delivered under the skin at the same time as the vaccine.

[…]

The researchers now plan to survey health care workers in developing nations in Africa to get input on the best way to implement this type of vaccination record keeping.

https://news.mit.edu/2019/storing-vaccine-history-skin-1218

Posted on

A tool to convert CSV, YML, XML, JSON and JSON-LD – oi v0.1

First release of oi is ready for testing. oi is an experimental command line tool to convert structured data from one format into another. The first release comes with rudimentary support for CSV, JSON, XML, YML and JSON-LD. The basic idea is to provide a straight in-out/out-in command that might not work completely correct but complements with other tools (jq, xsltproc,  sed, grep, awk, etc. ) to manipulate structured data on the command line.

Install

wget https://schnasse.org/deb/oi_0.0.1.deb
sudo apt install ./oi_0.0.1.deb # installs openjdk-11-jre

Example Usage

  1. Convert passwd to yml (use -t to print different formats)
oi /etc/passwd -d":" --header="login,password,uid,gid,comment,home,shell" -icsv
  1. List some disk info in yml (use -t to print different formats)
lsblk |grep -o sd.*|awk '{print $1 ";" $4 ";" $NF}'|oi -d";" --header="device,size,mount" -icsv
  1. Create adhoc Json-Ld Context for some arbitrary Json-API
curl https://api.github.com/users/jschnasse|oi -ijson -tcontext
Posted on

Command Line Tools: The growth of options

I found this table here: https://danluu.com/cli-complexity/

command 1979 1996 2015 2017
ls 11 42 58 58
rm 3 7 11 12
mkdir 0 4 6 7
mv 0 9 13 14
cp 0 18 30 32
cat 1 12 12 12
pwd 0 2 4 4
chmod 0 6 9 9
echo 1 4 5 5
man 5 16 39 40
which 0 1 1
sudo 0 23 25
tar 12 53 134 139
touch 1 9 11 11
clear 0 0 0
find 14 57 82 82
ln 0 11 15 16
ps 4 22 85 85
ping 12 12 29
kill 1 3 3 3
ifconfig 16 25 25
chown 0 6 15 15
grep 11 22 45 45
tail 1 7 12 13
df 0 10 17 18
top 6 12 14
Posted on

Java: Simple Read/Write of Maps as Json

You can use tools like Jackson to serialize and deserialize maps to json an vice versa.

Example

@Test
public void loadMapFromFileAndSaveIt(){
    Map<Object, Object> map = loadMap("map.json");
    map.put("8", "8th");
    map.remove("7");
    saveMap(map,"/path/to/map2.txt");
}

private Map<Object, Object> loadMap(String string) {
    ObjectMapper mapper = new ObjectMapper(); //should be initialized outside!
    try (InputStream in = Thread.currentThread().getContextClassLoader().getResourceAsStream("map.json")) {
        return mapper.readValue(in, HashMap.class);
    }catch (Exception e) {
        throw new RuntimeException(e);
    }
}

private void saveMap(Map<Object, Object> map,String path) {
    try (PrintWriter out = new PrintWriter(path)) {
        out.println(toString(map));
    } catch (Exception e) {
        throw new RuntimeException(e);
    }
}

public String toString(Object obj) {
    try (StringWriter w = new StringWriter();) {
        new ObjectMapper().configure(SerializationFeature.INDENT_OUTPUT, true).writeValue(w, obj);
        return w.toString();
    } catch (Exception e) {
        throw new RuntimeException(e);
    }
}

If the file map.json on your classpath contains

{
 "1":"1th",
 "2":"2th",
 "3":"3th",
 "4":"4th",
 "5":"5th",
 "6":"6th",
 "7":"7th"
}

The code above will modify it and writes it to a file /path/to/map2.txt that will contain the modified map.

{
  "1" : "1th",
  "2" : "2th",
  "3" : "3th",
  "4" : "4th",
  "5" : "5th",
  "6" : "6th",
  "8" : "8th"
}

 

Posted on

Unix tools introduced. Today: rsync

rsync is a very cool tool that can be used to copy files between hosts or between directories on the same host. Like the term ‘sync’ suggests the copy process can be controlled into great detail to modulate rsync’s behavior.  Take a look at the available options under: https://linux.die.net/man/1/rsync

This is my list of cool options.  I start with the most basic usage. The following command can be used to copy, and later on sync two directories.

rsync -avn /source/dir /target/dir

The command ‘archives’ file attributes (-a) and displays some status info (-v).

In the given form, the command only does a dry-run (-n). To execute the command remove the -n.

The command uses the short form of --archive (-a) which translates to (-rlptgoD).

  • -r – recursive copy
  • -l – copy symlinks as symlinks
  • -p – set target permissions to be the same as the source
  • -t – set target mtime to be the same as the source. Use this to support fast incremental updates based on mtime.
  • -g – set target group to be the same as the source
  • -o – set target owner to be the same as the source
  • -D – if remote user is superuser this recreates devices and other special files.

More cool options

Move

--remove-source-files This will remove copied files from source.

Update

--update This forces rsync to skip any files which exist on the destination and have a modified time that is newer than the source file.

Delete

--delete Delete files on target that do not exist in source tree.

Backup

--backup Make a backup of modified or removed files on target.

--backup-dir=date +%Y.%m.%d Specify a backup dir on target.

What to copy?

--min-size=1 Do not copy empty files. This can be particularly interesting if you have corrupted files in the source.

--max-size=100K Copy only small files. Can be used to handle small and large files differently.

--existing Only override files that already exist on the target. Do not create new files on target.

--ignore-existing Only copy files that do not exist on target.

--exclude-from Define excludes in a file.

Scheduling, Bandwidth and Performance

--time-limit Ends rsync after a certain time limit.

--stop-at=y-m-dTh:m Ends rsync at a specific time.

--partial Allows partial copies in case of interruptions.

--bwlimit=100 Limits bandwidth Specify KBytes/second. Good option if transfer of large files is required.

Output

  • -h output numbers in a human-readable format.
  • --progress display progress.
  • -i log change info.
  • --log-file= define a log file.
  • --quiet no output.
  •  -v Output status info. You can add more ‘v’.
  • Forgot to log any progress info? Use the following command to see what rsync is about to do. 
     ls -l /proc/$(pidof rsync)/fd/*

 

Posted on

Erfolgreiche Methoden in der Softwareentwicklung

Heute möchte ich einmal ein paar Methoden in der Softwarentwicklung vorstellen, die ich seit Jahren erfolgreich verfolge. Leider sind viele der vorgestellten Methoden ziemlich unbekannt. Aber es existiert ja auch viel schlechte Software….

Rabbit Prototyping

Beim Rabbit Prototyping geht es darum, möglichst schnell Fortschritte zu erzielen und sei es nur für kurze Zeit. Die Fortschrittskurve, die dabei entsteht, könnte von einem hüpfenden Kaninchen stammen, daher der Name. Vorteile des Rabbit Prototypings: Die meiste Zeit geht es voran. Alle haben etwas zu tun. Kaninchen sind putzig.

Bugfree Developement

Viele Softwareteams führen komplizierte Methoden ein oder heuern gar weitere Teams an um Qualitätsicherung und ausgeklügelte Testverfahren zu implementieren. Dies ist alles nicht nötig. Mit Bugfree Development lässt sich eine Menge Geld sparen. Eigentlich braucht man die Entwickler nur zu bitten, ab sofort keine Bugs mehr zu produzieren. Oft reicht eine klare Ansage  um von den Vorteilen des Bugfree Developments zu profitieren. Wer einmal bugfree entwickelt hat, wird es nicht mehr missen wollen. Vieles ist einfach Einstellungssache!

Deadly Standup

Kleine Teams sind oft effektiver. Aber wie hält man das Team klein? Diese kompetitive Erweiterung zu Scrum wirkt Wunder: Die Teammitglieder stellen sich im Kreis auf. Wer zuletzt steht hat gewonnen. Beim Deadly Standup geht es ausnahmsweise einmal nicht darum die Arbeit jedes einzelnen zu erleichtern. Stattdessen sagen sich alle mal so richtig die Meinung. Da darf es auch ruhig mal persönlich werden. Wer will darf das Standup verlassen, ist aber dann der Looser.