Software Development

Solving HttpMessageNotWritableException, Could not write JSON: Could not set field value in Spring Boot

(October 16, 2017) My experience of programming on the Spring platform is that you'll be coding along, everything is going great, until an inscrutible complex error message emerges from the depths of who knows where. This message seemed to involve some module or another having created a round peg to fit into a square hole. That is, I'd sent a GET request to retrieve a large object tree, and it seemed that while filling data into a field of a subsidiary object Spring/Hibernate/etc chose the wrong data type, and it threw up an error. The specific type of error was receiving no discussion anywhere, so it was impossible to see how others had solved the problem. The documentation was of no use in explaining anything about this. But, after a couple days of trials and errors and more errors, a solution was found. And, indeed, Spring/Hibernate had chosen the wrong data type to fill into one of the fields, and the solution is fairly simple.

Persisting complex Embeddable/Embedded objects in Spring/Hibernate

(October 8, 2017) Hibernate and Spring and the Java Persistence packages make for a powerful combination. You can define a complex tree of objects, and the system automagically creates the database schema, and automagically saves and retrieves that object tree from the database. It means you can innocently code away, until one day you try to include an Embedded object but it silently fails to be saved to the database, Hibernate silently fails to create database columns for that object, and when retrieving the object for which the Embedded object is a field, the object is null. No error messages are printed in the log. Instead you're left wondering, why didn't Hibernate persist this object? What's presented here is a solution using EntityListener methods to convert values in and out of shadow values stored in the Entity.

A fix for "PersistentObjectException - detached entity passed to persist" in Spring Boot

(September 6, 2017) Error messages coming out of Spring can be more than inscrutible. As discussed in my solution to transient unsaved properties, I'm defining a REST API using Swagger/OpenAPI targeting a Spring Boot implementation. The object models are complex, with lots of nested objects. Today's inscrutible message, "detached entity passed to persist", doesn't make much sense as a phrase does it? Detached? Detached Entity? Say what? Turns out it's not that hard a problem to fix.

A fix for org.hibernate.TransientPropertyValueException in Spring Boot

(September 1, 2017)

Error messages coming out of Spring can be more than inscrutible. In this case I'm defining a REST API using Swagger/OpenAPI targeting a Spring Boot implementation. The object models are complex, with lots of nested objects. It turns out you can POST a big blob of JSON with all kinds of structure, and Spring/JPA/Hibernate will swallow the whole thing and automatically parcel everything out to the corresponding database table. If you define the JPA annotations correctly, that is.

Get an annotation incorrect, and you'll be left scratching your head over what the heck you're being told by the error message. The Spring Team promises on a huge stack of bibles that Spring Boot automatically takes care of zillions of things for you, and it's so easy to write your code. In my experience it's instead a matter of papering complexity over complexity over a system containing about a thousand kitchen sinks worth of excess capabilities. There's no amount of papering-over that kind of complexity that will make the system simpler. But, the boss-man says to use Spring Boot, so here goes.

Deploying Docker images to a server without using a Docker Registry

(August 30, 2017)

We formerly deployed server applications to a Linux server using manual processes. An advanced team might use shell scripts to automate deployment. Over time tools like Chef or Ansible and more grew to handle ever-more-complex server application deployment scenarios. A few years ago, Docker came onto the scene with a whole new approach involving building a "Container" housing a complete operating system image that runs your application. Having built the Container, it's easy to ship that container to a server or run it on your laptop. The compelling gain is having the exact same development environment on your laptop as is deployed to your servers. Using the EXACT same environment streamlines your work by removing a ton of potentially destabilizing variables.

The preferred method is to build a Docker container image on your laptop, or on a build server, and upload the image to a Docker Registry. The image can then be downloaded from the Registry onto any number of systems.

What if you don't want to, or cannot, use a Registry? You could instead deploy the source code to the server, and build the container image on the server. That's a very unwise move, and it's better to ship the container image to the server. Turns out that is easy to do.

CSS Grid Changes Everything

(August 9, 2017)

Getting web content to layout as you want usually results in messy HTML and messy CSS. Limitations of both usually means you're nesting DIV's to get a structure where you can use various arcane CSS paradigms like absolute or relative positioning. CSS Grids changes all that. You can use simple HTML markup accompanied by straight-forward CSS rules to lay it out. It's a breath of fresh air. It's supported in ALL the modern web browsers.

Why not flexbox? Isn't that the thing that's supposed to be our page layout savior? The problem with flexbox is it runs in only one direction. For more complex layouts you need to nest flex containers. CSS Grid is a two-dimensional layout system and can handle more complex layouts.

Traditional CSS methodology, as well as flexbox, works from the content outward. CSS Grid works from the layout inward.

Throwing away local changes in Git workspace, allowing 'git pull' to proceed

(August 8, 2017)

Git is a powerful source code revision control system, letting teams large and small track changes between team members. It is widely used and very flexible. However, it's easy to get into an inscrutable state where it's not clear how to proceed.

Just now I found myself wishing to update my local repository with changes from co-workers. The repository contains UML models designed with Enterprise Architect, hence the primary thing in the repository is a large binary file. I'd been running Enterprise Architect on my local file, and there must have been some changes made to that file. As such Git told me there were local changes which I must merge before I could fetch the upstream changes. (specifically: Your local changes to the following files would be overwritten by merge) But I knew any change that had been made was unimportant, unneeded, and should be tossed away. Leaving me wondering how to throw away changes to a local Git repository.

Create a Spring Boot REST API using Swagger/OpenAPI "Generate Swagger OpenAPI REST API documentation for Spring Boot application"

(July 31, 2017)

The Swagger tools, and the OpenAPI format, are an excellent way to document REST API's and even to generate client or server stub libraries to ease implementation. The technology serves two purposes -- a) standardized documentation for REST API's, b) generating code from API documentation in several programming languages. An OpenAPI file is fairly simple to write, you declare REST endpoints, describe the parameters and the request type, and then describe responses. It allows you to define complex object models that can be used either as input to a service, or its output.

Unfortunately the Swagger website doesn't have adequate documentation of using the tools. And it proved difficult to find clear straight-forward tutorials showing how to get started. Even the most powerful tool can be hampered if folks are unable to use it.

The following tutorial is a complete demonstration of, starting from scratch, developing a small Spring Boot service using OpenAPI and the Swagger tools. We show how to go from an OpenAPI spec to generated Spring Boot code, and also how to generate an OpenAPI spec from running Spring Boot code. There are several issues with the workflow of generating code from the OpenAPI spec. It's more effective to instead write the service code, and add in the annotations required for the Swagger tools to generate the OpenAPI spec for you.

With the OpenAPI spec it's easy to produce interactive API documentation that programmers can try out directly in their web browser.

Correct character encoding with DOMDocument implementing a Wordpress content filter

(July 6, 2017) Using DOMDocument in a Wordpress content filter lets you correctly manipulate the content as HTML. It might be that Wordpress filters are commonly using regular expressions or text search/replace functions. While that can be fast and powerful, correctly changing HTML elements requires an HTML-oriented API. The specifics of HTML elements are such that regular expressions and text search/replace functions just don't cut it due to the many pitfalls from highly specific details of HTML. With DOMDocument you simply load the HTML into the library, then you use DOM functions to manipulate the HTML, then you serialize the DOM to HTML text, and voila powerful HTML manipulations easily performed. Unfortunately that method comes with its own pitfalls you must be careful of.

How to easily edit a Swagger/OpenAPI API specification using free tools

(June 15, 2017) The official way to edit a Swagger/OpenAPI document is using the Swagger Editor available through the swagger.io website, or to use the SwaggerHub website. The Editor can be run on your laptop inside a web browser which allows local JavaScript execution. As browsers tighten the screws on security the ability to do that may cease, and I see in the issue queue a request to make Electron versions of the Swagger tools. The SwaggerHub website is another option, especially as it offers lots of interesting features (at a fee). But, since OpenAPI is a free and open specification it's possible for others to develop tools. In this post we'll go over using the Swagger Editor, and using Atom to edit an OpenAPI specification.

Best implementation of the Swagger UI tool for your OpenAPI based Spring Boot REST server

(June 15, 2017)

Among the Swagger Tools is Swagger UI, an excellent tool for browsing an OpenAPI specification, the methods defined in the specification, the data models used in the specification. Swagger UI even lets you interact with the service from the documentation. It's a most useful take on "documentation", and to large measure is the whole reason for adopting OpenAPI/Swagger.

Unfortunately implementing the springfox-swagger-ui plugin in a Spring Boot Swagger application gives you a Swagger UI implementatation that's nowhere near as nice/capable as the demo app you see at petstore.swagger.io. There is zero documentation on how to implement the improved Swagger UI via the springfox-swagger-ui plugin. Instead, after quite a bit of experimentation, it seems one must turn to the Swagger UI Docker container.

Introduction to Time Series and InfluxDB

(May 30, 2017) Influx DB is an easy-to-use time-series database, that uses a familiar query syntax, allows for regular and irregular time series, and is part of a broad stack of platform components. This video goes over what Time Series Data is, a comparison of different Time Series Databases, and more.

Time-series data with InfluxDB, overview

(May 17, 2017) Time-series databases are used for scenarios involving collecting time-stamped data. You probably want to generate summaries (rollups and aggregations), data retention policies, and so forth. That every datum is time-stamped makes it immediately different from regular SQL databases. InfluxDB is an open source time-series database.

Easily use Let's Encrypt to HTTPS-protect your own server, for free

(2017-04-11 14:47 PDT) The search engines and browser makers are telling us to encrypt all websites. A driving factor is to protect everyone from not only miscreants wanting to hijack the web for nefarious goals, but the government security agencies who are snooping into everything. If everything on the Web is encrypted, then we'll all be better off. Until Let's Encrypt came along, the requirement to encrypt carried with it a high cost of paying for SSL certificates, and therefore many website owners would be unable to keep going. The free Lets Encrypt service opens HTTPS up to regular folk, allowing all website owners to encrypt their web traffic irregardless of how deep their pockets are. With that in mind, let's look into what it takes to set up HTTPS using Let's Encrypt.

SwaggerHub 101 An Introduction to Getting Started with SwaggerHub

(March 3, 2017) As the title suggest, this shows how to use SwaggerHub in a software development team. SwaggerHub covers a full life-cycle of API development in a team. Each project or "Organization" in SwaggerHub can have members and various settings. It can automatically generate a "mock API server" to try out the API directly via SwaggerHub.

Using Docker to host ARM toolchain to cross-compile C code

(2017-01-19 16:27) I'm starting up a project that will see me doing custom software development for an ARM single-board-computer running Linux. The recommendation isn't to do compiles ON the board, but instead to cross compile from a Linux workstation (Debian). But, I use a Mac laptop, as do most software engineers these days. While I could run VirtualBox to set up a Debian cross-compiling environment, Docker is much lighter weight. While Docker was originally targeted for deploying server applications, it is useful for packaging anything. In this case there's a ready-made set of Docker containers for cross-compilation including for ARM CPU's.

Swagger tutorial -- How to add Swagger OpenAPI to your REST API documentation

(Jan 9, 2017) An introduction to Swagger/OpenAPI. A core idea given is documentation that's interactive, and unlike typical documentation as it appears in a slide deck. The power of having an API described in an industry-standard format is the wide variety of tools available.

Building an API with Swagger

(Jan 1, 2017) Tutorial about creating an API using Swagger.

Make your own Raspberry Pi git repository server with Gogs and Docker

(2016-10-09 13:38) The Raspberry Pi is an amazing little computer that, while it's targeted at the DIY Hardware Maker, it is a full-fledged Linux computer that can be used to run services that used to require much bigger and more expensive computers. How long ago were office servers required to be $4000 systems from the likes of Dell Computers? It seems that the Raspberry Pi (and other tiny computers) can perform the same tasks at a low cost with minuscule energy requirements. To this end I'm setting up Gogs (a github-like server for Git repositories) on a Raspberry Pi. As I worked on the project it seemed most straightforward to use Docker to manage the Gogs process, and therefore the project became setting up Docker on Raspberry Pi to run other services.

Swagger and Open API Spec @codecampnyc 2016

(Oct 9, 2016)

Moving from "cloud-first" to "cloud-only" and therefore "API's" are more important than ever. Concept of the API Economy. Hence necessity for a better method to design REST-ful API's, and help others to use those API's. Another problem is the "Distributed Monolith" which is interlocking dependencies between microservices. Swagger is the solution.

Language agnostic. Helps to generate excellent documentation. No guesswork. Clearly documented REST API's.

Make a bash script detect the directory it's stored in, to access data there

(2016-09-19 15:52)

You may need to write a bash shell script that accesses data stored alongside the script, while your current working directory might be elsewhere. In my case the shell script needed to use Node.js scripts stored next to the shell scripts -- the shell script acting to simplify running the Node.js scripts. The "data" to be accessed in this case is the Node.js scripts, plus the support modules required to run them. You may have other data like a list of hostnames or who-knows-what.

InfluxDB CLI and Configuration Options

(Jul 27, 2016) Influx DB is an easy-to-use time-series database, that uses a familiar query syntax, allows for regular and irregular time series, and is part of a broad stack of platform components. The TICK stack is a set of four components that together make it quick and easy to collect time series data and graph it for users. One uses the CLI because it's always there, and is much easier than using CURL on the REST API.

Fixing 'Enter passphrase for /dev/fd/63' in a Gitlab CI job

(2016-06-30 16:53) If you're a Gitlab user you're probably hoping to use Gitlab CI to automate builds and deployments. You probably want to deploy something using rsync, using an SSH key for security. Unfortunately (in my opinion) the official Gitlab documentation is confusing. While the Gitlab team does provide example .gitlab-ci.yml files that are supposed to work, the actual specifics of what to do are sketchy, and I found myself puzzling over a curious error message: "Enter passphrase for /dev/fd/63" ... WTF?

Create hierarchical navigation for custom post types in Wordpress with PODS

(2015-06-20 23:27) On (longtailpipe.com) my Wordpress site I want to create "documentation" areas with a cluster of pages organized as a hierarchy, with an index to those pages showing as a sidebar. Many think of doing this for "Product Documentation", but I simply want to organize notes and additional material related to some books that I'm writing. Just as a book is organized by chapters and subsections, I want to hierarchically organize these pages.

How to customize the per-post authoring byline in Wordpress

(Mon Apr 06 2015 17:00:00 GMT-0700 (PDT))

CodeAnywhere - fast and flexible in-browser developers code editor

(2014-04-17 12:30:58 -0700) Codeanywhere is a code editor in a browser with an integrated ftp client, and all popular web formats are supported (HTML, PHP, JavaScript, CSS, and XML).

Converting a MySQL enum for use in SQLite3

(2012-01-06 21:11) I've got a database & website I want to move from using MySQL to using SQLite3.  Well, I think I want to use SQLite3.  Their document saying what sorts of uses make sense for SQLite3 are directly in line with my website, and I do want to remove some of the load off of my MySQL server so that it can have  cycles free for more important purposes.

However I've run into a couple troubles converting the schema so that it fits within SQLite3's limited SQL support.  Turns out that it doesn't support some column types and indexes.  And that the SQL produced by mysqldump contains some MySQLisms which SQLite3 just doesn't understand.

Implementing MySQL style AUTOINCREMENT in SQLite3

(Fri Jan 06 2012 16:00:00 GMT-0800 (PST))

Multiple headed Mercurial problems

(2008-11-20 20:37) Supposedly people who imbibe mercury become crazy as, well, a madhatter (so named because old-style hat-making practices involve the use of mercury). So maybe it sounds crazy to talk about multiple heads but in this case the mercury is the Mercurial source code management system. In Mercurial a 'head' is the endpoint of a chain of changes and it becomes downright inconvenient when there are multiple endpoints in the chain of changes in your source repository.

Using Mercurial patch queues

(Wed Nov 19 2008 16:00:00 GMT-0800 (PST))

Forbidding multiple heads in a shared mercurial repository

(Thu Oct 09 2008 17:00:00 GMT-0700 (PDT))
2016 Election Acer C720 Ad block AkashaCMS Amazon Amazon Kindle Amiga Android Anti-Fascism Apple Apple Hardware History Apple iPhone Apple iPhone Hardware April 1st Arduino ARM Compilation Astronomy Asynchronous Programming Authoritarianism Automated Social Posting Ayo.JS Bells Law Big Brother Big Finish Black Holes Blade Runner Blogger Blogging Books Botnet Botnets Cassette Tapes Cellphones Christopher Eccleston Chrome Chrome Apps Chromebook Chromebooks Chromebox ChromeOS CIA CitiCards Citizen Journalism Civil Liberties Clinton Cluster Computing Command Line Tools Computer Hardware Computer Repair Computers Cross Compilation Crouton Curiosity Rover Cyber Security Cybermen Daleks Darth Vader Data backup Data Storage Database Database Backup Databases David Tenant DDoS Botnet Detect Adblocker Developers Editors Digital Photography DIY DIY Repair DNP3 Docker Doctor Who Doctor Who Paradox Drobo Drupal Drupal Themes DVD E-Books E-Readers Early Computers Election Hacks Electric Bicycles Electric Vehicles Electron Emdebian Energy Efficiency Enterprise Node EPUB ESP8266 Ethical Curation Eurovision Event Driven Asynchronous Express Facebook Fake News Fedora VirtualBox File transfer without iTunes FireFly Fraud Freedom of Speech Gallifrey git Gitlab GMAIL Google Google Chrome Google Gnome Google+ Government Spying Great Britain Heat Loss Hibernate Home Automation HTTPS I2C Protocol Image Analysis Image Conversion Image Processing ImageMagick InfluxDB Infrared Thermometers Insulation Internet Internet Advertising Internet Law Internet of Things Internet Policy Internet Privacy iOS Devices iPad iPhone iPhone hacking Iron Man Iternet of Things iTunes Java JavaScript JavaScript Injection JDBC John Simms Journalism Joyent Kindle Marketplace Lets Encrypt LibreOffice Linux Linux Hints Linux Single Board Computers Logging Mac OS Mac OS X MacOS X setup Make Money Online MariaDB Mars Matt Lucas MEADS Anti-Missile Mercurial Michele Gomez Micro Apartments Military Hardware Minification Minimized CSS Minimized HTML Minimized JavaScript Missy Mobile Applications MODBUS Mondas MongoDB Mongoose Monty Python MQTT Music Player Music Streaming MySQL NanoPi Nardole NASA Net Neutrality Node Web Development Node.js Node.js Database Node.js Testing Node.JS Web Development Node.x North Korea Online advertising Online Fraud Online Journalism Online Video Open Media Vault Open Source Governance Open Source Licenses Open Source Software OpenAPI OpenVPN Personal Flight Peter Capaldi Photography PHP Plex Plex Media Server Political Protest Postal Service Power Control Privacy Production use Public Violence Raspberry Pi Raspberry Pi 3 Raspberry Pi Zero Recycling Remote Desktop Republicans Retro-Technology Reviews Right to Repair River Song Robotics Rocket Ships RSS News Readers rsync Russia Russia Troll Factory SCADA Scheme Science Fiction Season 1 Season 10 Season 11 Security Security Cameras Server-side JavaScript Shell Scripts Silence Simsimi Skype Social Media Warfare Social Networks Software Development Space Flight Space Ship Reuse Space Ships SpaceX Spear Phishing Spring Spring Boot SQLite3 SSD Drives SSD upgrade SSH SSH Key SSL Swagger Synchronizing Files Telescopes Terrorism The Cybermen The Daleks The Master Time-Series Database Torchwood Total Information Awareness Trump Trump Administration Ubuntu UDOO Virtual Private Networks VirtualBox VLC VNC VOIP Web Applications Web Developer Resources Web Development Web Development Tools Web Marketing Website Advertising Weeping Angels WhatsApp Window Insulation Wordpress YouTube