Unix tools introduced. Today: yes

The unix tool yes provides one simple yet very useful functionality. It repeatedly  prints ‘yes’ (or any other string) to the console.

This can be used to automatically answer questions in scripted environments.

Example:

yes|sudo rpm -ivh mysql-community-release-el7-5.noarch.rpm

 

Converting XML to JSON

You can use standard tools for it

1. Use the tool xjc from your jdk to generate Java classes from schema

Since Java 9 you must explicitly add JAXB as module with –add-modules java.se.ee See: How to resolve java.lang.NoClassDefFoundError: javax/xml/bind/JAXBException in Java 9

Since Java 11 you have to download xjc in an extra step from https://javaee.github.io/jaxb-v2/

2. Read in as XML write out as JSON using Jackson

Example

With https://schema.datacite.org/meta/kernel-4.1/metadata.xsd

1. Use the tool xjc from your jdk

In my example I will use a fairly complex example based on datacite schemas.

/path/to/jdk/bin/xjc -d /path/to/java/project \
-p stack24174963.datacite \
 https://schema.datacite.org/meta/kernel-4.1/metadata.xsd

This will reply with

parsing a schema...
compiling a schema...
stack24174963/datacite/Box.java
stack24174963/datacite/ContributorType.java
stack24174963/datacite/DateType.java
stack24174963/datacite/DescriptionType.java
stack24174963/datacite/FunderIdentifierType.java
stack24174963/datacite/NameType.java
stack24174963/datacite/ObjectFactory.java
stack24174963/datacite/Point.java
stack24174963/datacite/RelatedIdentifierType.java
stack24174963/datacite/RelationType.java
stack24174963/datacite/Resource.java
stack24174963/datacite/ResourceType.java
stack24174963/datacite/TitleType.java
stack24174963/datacite/package-info.java

If you look into Resource.Creator and Resource.Contributor you will see that the member variables givenName and familyName are not correctly typed. Change their type from Object to String, also apply your changes to the corresponding getter and setter methods!

2. Read in as XML write out as JSON using Jackson

import com.fasterxml.jackson.databind.ObjectMapper;
  import com.fasterxml.jackson.databind.SerializationFeature;

  import stack24174963.datacite.Resource;

  public class HowToXmlToJsonWithSchema {
    @Test
    public void readXmlAndConvertToSchema() throws Exception {
        String example = "schemas/datacite/kernel-4.1/example/datacite-example-complicated-v4.1.xml";
        try (InputStream in = Thread.currentThread().getContextClassLoader().getResourceAsStream(example)) {
            Resource resource = JAXB.unmarshal(in, Resource.class);
            System.out.println(asJson(resource));
        }
    }

    private String asJson(Object obj) throws Exception {
        StringWriter w = new StringWriter();
        new ObjectMapper().configure(SerializationFeature.INDENT_OUTPUT, true).writeValue(w, obj);
        String result = w.toString();
        return result;
    }
  }

Prints:

{
  "identifier" : {
    "value" : "10.5072/testpub",
    "identifierType" : "DOI"
  },
  "creators" : {
    "creator" : [ {
      "creatorName" : {
        "value" : "Smith, John",
        "nameType" : "PERSONAL"
      },
      "givenName" : "John",
      "familyName" : "Smith",
      "nameIdentifier" : [ ],
      "affiliation" : [ ]
    }, {
      "creatorName" : {
        "value" : "つまらないものですが",
        "nameType" : null
      },
      "givenName" : null,
      "familyName" : null,
      "nameIdentifier" : [ {
        "value" : "0000000134596520",
        "nameIdentifierScheme" : "ISNI",
        "schemeURI" : "http://isni.org/isni/"
      } ],
      "affiliation" : [ ]
    } ]
  },
  "titles" : {
    "title" : [ {
      "value" : "Właściwości rzutowań podprzestrzeniowych",
      "titleType" : null,
      "lang" : "pl"
    }, {
      "value" : "Translation of Polish titles",
      "titleType" : "TRANSLATED_TITLE",
      "lang" : "en"
    } ]
  },
  "publisher" : "Springer",
  "publicationYear" : "2010",
  "resourceType" : {
    "value" : "Monograph",
    "resourceTypeGeneral" : "TEXT"
  },
  "subjects" : {
    "subject" : [ {
      "value" : "830 German & related literatures",
      "subjectScheme" : "DDC",
      "schemeURI" : null,
      "valueURI" : null,
      "lang" : "en"
    }, {
      "value" : "Polish Literature",
      "subjectScheme" : null,
      "schemeURI" : null,
      "valueURI" : null,
      "lang" : "en"
    } ]
  },
  "contributors" : {
    "contributor" : [ {
      "contributorName" : {
        "value" : "Doe, John",
        "nameType" : "PERSONAL"
      },
      "givenName" : "John",
      "familyName" : "Doe",
      "nameIdentifier" : [ {
        "value" : "0000-0001-5393-1421",
        "nameIdentifierScheme" : "ORCID",
        "schemeURI" : "http://orcid.org/"
      } ],
      "affiliation" : [ ],
      "contributorType" : "DATA_COLLECTOR"
    } ]
  },
  "dates" : null,
  "language" : "de",
  "alternateIdentifiers" : {
    "alternateIdentifier" : [ {
      "value" : "937-0-4523-12357-6",
      "alternateIdentifierType" : "ISBN"
    } ]
  },
  "relatedIdentifiers" : {
    "relatedIdentifier" : [ {
      "value" : "10.5272/oldertestpub",
      "resourceTypeGeneral" : null,
      "relatedIdentifierType" : "DOI",
      "relationType" : "IS_PART_OF",
      "relatedMetadataScheme" : null,
      "schemeURI" : null,
      "schemeType" : null
    } ]
  },
  "sizes" : {
    "size" : [ "256 pages" ]
  },
  "formats" : {
    "format" : [ "pdf" ]
  },
  "version" : "2",
  "rightsList" : {
    "rights" : [ {
      "value" : "Creative Commons Attribution-NoDerivs 2.0 Generic",
      "rightsURI" : "http://creativecommons.org/licenses/by-nd/2.0/",
      "lang" : null
    } ]
  },
  "descriptions" : {
    "description" : [ {
      "content" : [ "\n      Lorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam nonumy eirmod tempor invidunt ut labore et dolore magna aliquyam erat, sed diam voluptua. At vero eos et accusam et justo duo dolores et ea rebum. Stet clita kasd gubergren, no sea\n      takimata sanctus est Lorem ipsum dolor sit amet. Lorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam nonumy eirmod tempor invidunt ut labore et dolore magna aliquyam erat, sed diam voluptua. At vero eos et accusam et justo duo dolores\n      et ea rebum. Stet clita kasd gubergren, no sea takimata sanctus est Lorem ipsum dolor sit amet.\n    " ],
      "descriptionType" : "ABSTRACT",
      "lang" : "la"
    } ]
  },
  "geoLocations" : null,
  "fundingReferences" : null
}

For XML input:

<?xml version="1.0" encoding="UTF-8"?>
  <resource xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://datacite.org/schema/kernel-4" xsi:schemaLocation="http://datacite.org/schema/kernel-4 http://schema.datacite.org/meta/kernel-4.1/metadata.xsd">
    <identifier identifierType="DOI">10.5072/testpub</identifier>
    <creators>
      <creator>
        <creatorName nameType="Personal">Smith, John</creatorName>
        <givenName>John</givenName>
        <familyName>Smith</familyName>
      </creator>
      <creator>
        <creatorName>つまらないものですが</creatorName>
        <nameIdentifier nameIdentifierScheme="ISNI" schemeURI="http://isni.org/isni/">0000000134596520</nameIdentifier>
      </creator>
    </creators>
    <titles>
      <title xml:lang="pl">Właściwości rzutowań podprzestrzeniowych</title>
      <title xml:lang="en" titleType="TranslatedTitle">Translation of Polish titles</title>
    </titles>
    <publisher>Springer</publisher>
    <publicationYear>2010</publicationYear>
    <subjects>
      <subject xml:lang="en" subjectScheme="DDC">830 German &amp; related literatures</subject>
      <subject xml:lang="en">Polish Literature</subject>
    </subjects>
    <contributors>
      <contributor contributorType="DataCollector">
        <contributorName nameType="Personal">Doe, John</contributorName>
        <givenName>John</givenName>
        <familyName>Doe</familyName>
        <nameIdentifier nameIdentifierScheme="ORCID" schemeURI="http://orcid.org/">0000-0001-5393-1421</nameIdentifier>
      </contributor>
    </contributors>
    <language>de</language>
    <resourceType resourceTypeGeneral="Text">Monograph</resourceType>
    <alternateIdentifiers>
      <alternateIdentifier alternateIdentifierType="ISBN">937-0-4523-12357-6</alternateIdentifier>
    </alternateIdentifiers>
    <relatedIdentifiers>
      <relatedIdentifier relatedIdentifierType="DOI" relationType="IsPartOf">10.5272/oldertestpub</relatedIdentifier>
    </relatedIdentifiers>
    <sizes>
      <size>256 pages</size>
    </sizes>
    <formats>
      <format>pdf</format>
    </formats>
    <version>2</version>
    <rightsList>
      <rights rightsURI="http://creativecommons.org/licenses/by-nd/2.0/">Creative Commons Attribution-NoDerivs 2.0 Generic</rights>
    </rightsList>
    <descriptions>
      <description xml:lang="la" descriptionType="Abstract">
        Lorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam nonumy eirmod tempor invidunt ut labore et dolore magna aliquyam erat, sed diam voluptua. At vero eos et accusam et justo duo dolores et ea rebum. Stet clita kasd gubergren, no sea
        takimata sanctus est Lorem ipsum dolor sit amet. Lorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam nonumy eirmod tempor invidunt ut labore et dolore magna aliquyam erat, sed diam voluptua. At vero eos et accusam et justo duo dolores
        et ea rebum. Stet clita kasd gubergren, no sea takimata sanctus est Lorem ipsum dolor sit amet.
      </description>
    </descriptions>
  </resource>

 

See also:

https://stackoverflow.com/a/49977184/1485527

and

https://github.com/jschnasse/overflow/tree/master/src/test/java/stack24174963

Process parts of XML with XPath in Java

This shows you how to

1. Read in an XML file to a DOM
2. Filter out a set of Nodes with XPath
3. Perform a certain action on each of the extracted Nodes.

We will call the code with the following statement

processFilteredXml(xmlIn, xpathExpr,(node) -> {/*Do something...*/;});

In our case we want to print some creatorNames from a book.xml using “//book/creators/creator/creatorName” as xpath to perform a printNode action on each Node that matches the XPath.

Full code

@Test
public void printXml() {
    try (InputStream in = readFile("book.xml")) {
        processFilteredXml(in, "//book/creators/creator/creatorName", (node) -> {
            printNode(node, System.out);
        });
    } catch (Exception e) {
        throw new RuntimeException(e);
    }
}

private InputStream readFile(String yourSampleFile) {
    return Thread.currentThread().getContextClassLoader().getResourceAsStream(yourSampleFile);
}

private void processFilteredXml(InputStream in, String xpath, Consumer<Node> process) {
    Document doc = readXml(in);
    NodeList list = filterNodesByXPath(doc, xpath);
    for (int i = 0; i < list.getLength(); i++) {
        Node node = list.item(i);
        process.accept(node);
    }
}

public Document readXml(InputStream xmlin) {
    try {
        DocumentBuilderFactory dbf = DocumentBuilderFactory.newInstance();
        DocumentBuilder db = dbf.newDocumentBuilder();
        return db.parse(xmlin);
    } catch (Exception e) {
        throw new RuntimeException(e);
    }
}

private NodeList filterNodesByXPath(Document doc, String xpathExpr) {
    try {
        XPathFactory xPathFactory = XPathFactory.newInstance();
        XPath xpath = xPathFactory.newXPath();
        XPathExpression expr = xpath.compile(xpathExpr);
        Object eval = expr.evaluate(doc, XPathConstants.NODESET);
        return (NodeList) eval;
    } catch (Exception e) {
        throw new RuntimeException(e);
    }
}

private void printNode(Node node, PrintStream out) {
    try {
        Transformer transformer = TransformerFactory.newInstance().newTransformer();
        transformer.setOutputProperty(OutputKeys.INDENT, "yes");
        transformer.setOutputProperty(OutputKeys.OMIT_XML_DECLARATION, "yes");
        transformer.setOutputProperty("{http://xml.apache.org/xslt}indent-amount", "2");
        StreamResult result = new StreamResult(new StringWriter());
        DOMSource source = new DOMSource(node);
        transformer.transform(source, result);
        String xmlString = result.getWriter().toString();
        out.println(xmlString);
    } catch (Exception e) {
        throw new RuntimeException(e);
    }
}

Prints

<creatorName>Fosmire, Michael</creatorName>

<creatorName>Wertz, Ruth</creatorName>

<creatorName>Purzer, Senay</creatorName>

For book.xml

<book>
  <creators>
    <creator>
      <creatorName>Fosmire, Michael</creatorName>
      <givenName>Michael</givenName>
      <familyName>Fosmire</familyName>
    </creator>
    <creator>
      <creatorName>Wertz, Ruth</creatorName>
      <givenName>Ruth</givenName>
      <familyName>Wertz</familyName>
    </creator>
    <creator>
      <creatorName>Purzer, Senay</creatorName>
       <givenName>Senay</givenName>
       <familyName>Purzer</familyName>
    </creator>
  </creators>
  <titles>
    <title>Critical Engineering Literacy Test (CELT)</title>
  </titles>
</book>

See also

https://stackoverflow.com/a/52736526/1485527

https://github.com/jschnasse/overflow/tree/master/src/test/java/stack52720162

CGN -Was wird aus der Nachtflugregelung?

Seit dem 11. Dezember ist Friedrich Merz der neue Vorsitzende des Aufsichtsrates am Flughafen Köln Bonn. Damit bekommt der nächtliche Fluglärm für mich ein Gesicht. Gratulation an die Leute vom Casting – Topbesetzung!

Derweil:

Die Lärmschutzgemeinschaft Flughafen Köln/Bonn e.V. fordert mittlerweile nur noch ein Nachtflugverbot (0-5Uhr) für Passagiermaschinen.

http://www.fluglaerm-koeln-bonn.de/index.php/was-wird-aus-der-nachtflugregelung/

Neu bei Google: jan.schnasse.org/blog

Obwohl die Seite nirgendwo verlinkt ist. Ein kurzer Blick auf die Referrer in Matomo zeigt an, dass die Seite ausschließlich per “Direct Entry” zugegriffen wurde.  Mit einer Suche nach “jan schnasse” ist sie dennoch auffindbar.

Auch Suchmaschinen wurden von Matomo bislang nicht gesichtet. Wie kommt die Seite nur in den Google Index? Vielleicht über einen, z.B. von Google Analytics, ausgewerteten Http-Referer beim Klick auf einen der ausgehenden Links? Google Ressourcen, z.B. Fonts, lade ich meines Wissens nicht. Meine Vermutung ist, dass die Youtube-Links mich verraten haben.