piątek, 24 marca 2023

Eliminate dreadful NPEs with NullAway

What is NullAway

Static code analysis has always been a little controversial in the Java development community. As with almost each and every tool, it can be used for good (to improve code quality) or bad (infuriate engineers with picky rules). 

Null Away (https://github.com/uber/NullAway) is however different. There are no rules to configure, no code styles to discuss. Just one plugin that will help eliminate NPEs in your code by reviewing all code execution paths.


Configuration


For Gradle

https://github.com/uber/NullAway#gradle


For Maven: 



<plugin>

        <groupId>org.apache.maven.plugins</groupId>

        <artifactId>maven-compiler-plugin</artifactId>

        <configuration>

          <source>${maven.compiler.source}</source>

          <target>${maven.compiler.target}</target>

          <fork>true</fork>

          <compilerArgs>

            <arg>-XDcompilePolicy=simple</arg>

            <arg>-Xplugin:ErrorProne -XepAllErrorsAsWarnings -XepExcludedPaths:.*/src/test/java/.* -Xep:NullAway:ERROR -XepOpt:NullAway:TreatGeneratedAsUnannotated=true -XepOpt:NullAway:ExcludedClasses=sf.cloudmetricsyncer.CloudProvider -XepOpt:NullAway:AnnotatedPackages=sf.cloudmetricsyncer,sf.externalmonitor</arg>

            <arg>-J--add-exports=jdk.compiler/com.sun.tools.javac.api=ALL-UNNAMED</arg>

            <arg>-J--add-exports=jdk.compiler/com.sun.tools.javac.file=ALL-UNNAMED</arg>

            <arg>-J--add-exports=jdk.compiler/com.sun.tools.javac.main=ALL-UNNAMED</arg>

            <arg>-J--add-exports=jdk.compiler/com.sun.tools.javac.model=ALL-UNNAMED</arg>

            <arg>-J--add-exports=jdk.compiler/com.sun.tools.javac.parser=ALL-UNNAMED</arg>

            <arg>-J--add-exports=jdk.compiler/com.sun.tools.javac.processing=ALL-UNNAMED</arg>

            <arg>-J--add-exports=jdk.compiler/com.sun.tools.javac.tree=ALL-UNNAMED</arg>

            <arg>-J--add-exports=jdk.compiler/com.sun.tools.javac.util=ALL-UNNAMED</arg>

            <arg>-J--add-opens=jdk.compiler/com.sun.tools.javac.code=ALL-UNNAMED</arg>

            <arg>-J--add-opens=jdk.compiler/com.sun.tools.javac.comp=ALL-UNNAMED</arg>

          </compilerArgs>

          <annotationProcessorPaths>

            <path>

              <groupId>com.google.errorprone</groupId>

              <artifactId>error_prone_core</artifactId>

              <version>2.16</version>

            </path>

            <path>

              <groupId>com.uber.nullaway</groupId>

              <artifactId>nullaway</artifactId>

              <version>0.10.5</version>

            </path>

          </annotationProcessorPaths>

        </configuration>

      </plugin>


 

How to


For a new project it’s easy - just enable the plugin and fix the build each time it is required. For old code, especially with a large codebase, it might be trickier. NullAway plugin is not reporting all of the violations on the first run - only a subset, making the whole iterative process longer and more painful. 


Here’s a proposed approach I’ve successfully applied to a number of projects:


  1. Start with service boundaries - API, DB data model. Review which fields definitely need to be nullable. If possible implement appropriate validation to reject illegal NULLs or apply defaults.

  2. If you have a good modularization of the code - split work by sub-package, using configuration options “-XepOpt:NullAway:AnnotatedPackages=XYZ” and “-XepOpt:NullAway:UnannotatedSubPackages=ABC”

  3. Run the build and review  “returning @Nullable expression from method with @NonNull return type” messages. Consider if you really need to return NULLs in such cases. Perhaps a default value would be better.

  4. Review “initializer method does not guarantee @NonNull field XXX is initialized along all control-flow paths (remember to check for exceptions or early returns).” messages. Perhaps an instance of the class is initialized in a different way (guaranteed to be called after construction but before the first use)? If so, add a pre-configured (plugin configuration) initialization annotation to the method. This is also a great moment to think about making the data model (or parts of the model) immutable without any NULLs.

  5. Review messages pertaining the class hierarchy (super / subclass) 

    1. parameter X is @NonNull, but parameter in superclass method ABC#abc) is @Nullable"

    2. method returns @Nullable, but superclass method ABC#abc returns @NonNull

  6. Review “passing @Nullable parameter 'null' where @NonNull is required” - do you need to pass an explicit null? Perhaps a default (empty) value would be better?

  7. Review “passing @Nullable parameter XXX where @NonNull is required”. This often means that a data model class properties are nullable. Perhaps the model entity can be turned into immutable one, getting rid of NULLs there entirely?




Tips


Initialisation annotation

In some cases instance is initialized before the first use in some other way than via a constructor. This may happen in a typical framework implementing some kind of an instance lifecycle. In order to notify NullAway that fields initialization will happen in such method, one can use the initialization annotation:  https://github.com/uber/NullAway/wiki/Supported-Annotations#initialization 


NullSafeMap

Map get() is treated as inherently unsafe (nullable). In many cases however a programmer knows that map will contain mapping for a particular key. In order to use NullAway efficiently in such cases it’s a good idea to have an utility (eg NullSafeMap) with static nullSafeGet(Map, key) method annotated with @SuppressWarnings("NullAway") 


Use standards 

Out of possible annotations, I recommend: javax.annotation.Nullable. Keep it… standardized ;-)


Local variables

It’s not possible to suppress a check for a local variable - in such cases it’s best to extract a separate method (one liner) out of expression where the variable is used and annotate the method.


Generated code

One can easily ignore generated code (since they can’t do much about it) using: -XepOpt:NullAway:TreatGeneratedAsUnannotated=true 


Last resort - ignoring particular classes

Exclude some classes (only if really needed) using:  -XepOpt:NullAway:ExcludedClasses=XYZ



Now go out there, have fun coding and hunt down some nasty NullPointerExceptions! ;-)


czwartek, 12 sierpnia 2021

Snyk JVM Ecosystem Report 2021

...has just been released: https://snyk.io/jvm-ecosystem-report-2021 and gives quite interesting overview of the Java world we're living in. Full version (33 pages!) is free and available for download under this link .

Key takeaways

  1. AdoptOpenJDK dominates - 44% of developers uses this distro. Interesting point - Amazon Corretto gets only 9% share. Sounds as if the community learnt not to trust corporations ;-)
  2. Systems move away from Java 8 (more use 11 in production already). Also, more than 25% of the Java developers use Java 15 in development. That's a good thing showing that ecosystem is healthy and alive.
  3. Kotlin is the second most important language on the JVM (more popular than Groovy or Scala). There are also 531k repos in Kotlin on Github as compared to 194k in Scala / 64k in Groovy. One possible explanation is that Kotlin simply fixes what's broken in Java without overcomplicating things (yes Scala I'm looking at you). 

Most surprising (or are they?)

  1. Maven is still the most popular build system for Java with 76% of developers using it. There is more - Maven scored better numbers than a year ago! Given high popularity of Gradle here-and-there (for example in OSS projects) and general tendency to scrap XML-based tools, this is quite interesting piece of data.
  2. Domination of Spring, with more than 50% of developers using Spring Boot and almost 30% using Spring MVC. This sounds surprising as I've recently heard a lot (and I mean it - a lot) of complains about how bloated and complex Spring has become. With so many configuration modules being loaded automatically by default it's sometimes crazy hard to understand what the hell is happening when the app starts. On the other hand Spring Boot allows for startup of a new project in a mere second (or few more). Does that mean that in general developers prefer (or are required to prefer) speed of development versus quality/speed/control? 

Happy reading! Let me know if you find something surprising / interesting in the report.

środa, 22 kwietnia 2020

JSON stringify - transformation tip

JSON stringify


Well known JavaScript method for transforming objects or values into string. However, there is an optional second parameter to the call - a replacer. Once provided, alters the stringification process.

Method

If a method is provided, will be applied to each property of an object. Very handy tool if serialised form has to have for example different values. For example:


let data = {'one' : 1, 'two' : 'a string'};
JSON.stringify(data, (key, value) => key === 'one' ? 2 : value)
// "{"one":2,"two":"a string"}"


As the object being serialized is also available (as this reference), we may perform a wider range of transformations, for example:


data = {x : 1, y : 2, multiplier : 10};
function replacer(key, value) {return typeof value === 'number' ? value * this.multiplier : value}
JSON.stringify(data, replacer);
// "{"x":10,"y":20,"multiplier":100}"

Array

Simply put - a whitelist. Only selected properties will appear in the serialised form. For example:


let data = {'one' : 1, 'two' : 'a string', 'garbage' : 'some'};
JSON.stringify(data, ['one' ,'two']);
// "{"one":1,"two":"a string"}"


To sum up - a very handy tool for quick-and-dirty processing of a bare JSON / JS objects.

środa, 23 stycznia 2019

How to fix: Webpack file-loader corrupting fonts

The problem

Recently I've been forking on a fairly simple React / Webpack application, built with newest CRA (https://facebook.github.io/create-react-app/).
Everything was fine in the development but when I built a production package, I started noticing strange warnings in dev tools console. All of them concerned fonts loading, for example:

Failed to decode downloaded font
OTS parsing error: incorrect entrySelector for table directory
OTS parsing error: incorrect file size in WOFF header

After short investigation it became apparent that file-loader is not able to handle WOFF or TTF fonts correctly, returning empty content.


The solution

Webpack allows for configuration of loaders per file name pattern. Therefore it's fairly easy to switch fonts loading to use URL loader instead of file loader. Just add following right before generic file loader kicks in (typically - last entry  in module/rules/oneOf):

{
 test: [/\.woff$/,/\.ttf$/],

  loader: require.resolve('url-loader'),
options: { limit: 150000,
mimetype : 'application/font-woff',
name: 'file?name=static/media/[name].[hash:8].[ext]',
}, },

Now all of your fonts will be served as embedded data URLs. This obviously will make resulting package bigger and will prevent separate caching of fonts - it's a tradeof. Also please make sure you set appropriate limit so that all "offending" fonts are captured!

wtorek, 22 stycznia 2019

REST API cook book

Introduction


Recently I've stumbled upon an interesting site - REST Cook Book. As nowadays almost everyone develops RESTful APIs and there is a great deal of controversy regarding how to design it properly (say: singular vs plural nouns or collections range query) I decided to take a look at the "recipes" Author serves.

The recipes


First of all, the site is NOT a REST API design guide. Instead, Author decided to write a bunch of short articles concerning some interesting pieces of frequently asked questions. Second of all, Author seems to be focused on HATEOAS (Hypertext As The Engine Of Application State) which differs a bit from "JSON with RPC" style as commonly found in modern Java web applications. The most important difference is that HATEOAS does not care so much about URL structure, as each possible operation for a resource (in other words - possible transition from current state) is returned in form of link / rel / href.
Nevertheless it's an interesting read. Typically authors write a lot about the basics - like mentioned singular vs plurals or how to use basic HTTP methods. Here, we get recipes concerning such rarely discussed aspects as:

  • collection pagination - with link rel = next / first / last 
  • discovering supported HTTP methods with "OPTIONS" 
  • implementing asynchronous operations using 202 "ACCEPTED"
There is also a lot of discussion under almost every recipe, which is very resourceful.