Optional Shouldn’t Be Optional

The Optional Horror Show

I’ve been reading-up on Java 8’s Optional<T> class and it’s proper usage. It might not surprise you that opinions on its correct usage vary greatly and passionately.

  • Only for return values never arguments
  • Use it everywhere!
  • Never as a field value
  • Only for return values…but only sometimes
  • Not for all getters…but some
  • Let’s make it serializable!

Even the experts don’t have consensus. So why was optional introduced? As I understand it, its primary purpose was to make the stream API more fluent and to return a reasonable result from terminal stream operations. Terminal operations need to return a result even when a collection is empty. For example:

x = s.findFirst().or(valueIfEmpty)

Nicolai Parlog wrote an excellent article discussing the background of Optional. I suggest everyone read it. It gives a great summary of the back and forth that went on around the introduction of Optional<T>.

I will use Optional<T> but ultimately I think it was a misstep. My primary issue with Optional is that it is a wrapper class that must be instantiated. Because of this I have to decide when to use it and when not to use it. This will lead to many different styles of using Optional and to an overall proliferation of use. For example:

  • Do we use Set<Optional<T>> and Map<K, Optional<V>>? Some people will and some people wont.
  • Do we have Java Bean getters return Optional<T>? If the answer is “sometimes” then how is that not a problem?
  • Because Optional<T> is a wrapper class it cannot simply implement Serializable – if it did it would have indeterminate behavior when it wraps a non-serializable class.
  • Most of the existing Java SDK classes do not and will not use Optional<T>. So I will have to accommodate to styles anyway.

With Optional as a wrapper class so many frameworks have been impacted and thus need to change. For example:

  • JSON, XML, JPA, etc. libraries need to consider Optional in order to properly read and write values from getters that return an Optional<T>.
  • Bean Validation needs to unwrap Optional<T> and validate its value…but wait, what if we want @NotNull bean constraint on the Optional itself and not it’s wrapped value?
  • Existing Java SDK classes send an unclear message on optional. They do not use Optional<T> so now we have to take care ANYWAY when using return values.

The decision to introduce Optional as a wrapper class has a far-reaching impact and it’s usage will proliferate and, worse yet, proliferate in an inconsistent way. At this point I do not think the Genie can be put back into the lamp.

Do Alternatives Exist?

If you think about it, Optional is really just behavior it is not state. For example, Optional allows us to write fluent code like this:

String value =

Optional behavior does require any properties outside of the object it is operating on. Now, I am not a language designer. I assume everyone that worked on defining Lambda is much smarter than I am. Nonetheless, I cannot wrap my head around the decision to introduce the Optional<T> wrapper. Wouldn’t life have been easier (even if done as a hack) to allow all objects (including null) to accept methods like #ifPresent, #orElse, and #map? Wouldn’t making this behavior available to all variables solve many of the issues associated with making Optional<T> a wrapper class?

  • There would be no Optional class to ponder how best to use.
  • Stream terminal operations work fine.
  • There are no issues regarding whether it should be a field or not, serializable or not, etc.
  • Creation semantics between #ofNullable and #of
  • Existing frameworks that deal with beans wouldn’t need to change.
  • No need to accommodate older classes that do not return an Optional<T>.

If we go back to our earlier example, instead of this:

String value =

We would be able to always and safely write this:

String value = map.get("foo").orElse("bar");

No code needs to re-written to make use of optional behavior, you no longer have to complain that an Optional should be serializable, wonder where you should use a normal object or instantiate an Optional wrapper for that object, etc. And you would never wonder if you should declare a map as Map<String, Optional<String>>.

So you may wonder why the Java experts didn’t make null and java.lang.Object respond to the Optional<T> methods. Well this idea does have at least one VERY BIG issue. If your code (or any code you depended on) happened to contain a class with a method named #ifPresent, #orElse, or #map you’d be screwed – especially if any of those methods had an incompatible signature. Your code would not compile. Wow…that sucks, built-in Optional methods seemed like such a simple solution.

Is there a way around this issue? Well, some languages use an operator instead of methods for optional behavior and that could get us past part of the problem. It’s sometimes called an Elvis Operator or Null Coalescing Operator. For example, in Groovy or PHP you might write the above code like this:

String value = map.get("foo") ?: "bar";

You can think of it as shorthand for:

String value = map.get("foo") != null ? map.get(“foo”) : "bar";

That works great for #orElse, but it doesn’t give us that cool #ifPresent functionality. For example:


With an Elvis operator you’d have to do something like this:

dbRecord.setUserName(json.getUserName() ?: dbRecord.getUserName());

As you can see the two variants are not functionally the same. #ifPresent allows us to complete forgo calling dbRecord#setUserName when json#getUserName is not present. So I suppose we’d need something more than a null coalescing operator to get rid of the Optional<T> class but perhaps for Java 8 that one operator would have been enough.

The JDK 8 team had many discussions about this, so clearly there is no perfect solution. Here are some ideas I’m sure they threw around:

  1. Make all objects (including null) respond to #ifPresent, #orElse, and #map and let the chips fall where they may. If your code or a library has a class that uses these methods then you get a compilation error. You either can’t use the latest version of Java or you have to update your code and/or libraries.
  2. Make a special message send operator for these built-in and universal methods (system messages). For example, value:ifPresent or value->ifPresent. Ew. I don’t like this idea but I still like it better than I like instantiating Optional wrapper objects.
  3. Create an Elvis operator and leave out friends like #isPresent and #map. Again, not my favorite choice but still better than instantiating Optional<T>

My preference is #1. But hey, I’m not responsible for the adoption of an entire language upgrade. I wouldn’t get the heat for any broken code. Then again, most popular libraries respond quickly to changes. If I think about it, the slight chance you may have to remove an existing #ifPresent, #orElse, or #map from a library is probably less effort than deciding what to do if a Java Bean has a getter that returns an instance of Optional<T>. And code that is mine can usually be fixed quickly. Most issues could be determined at compile time. I don’t think there are a TON of classes that have these methods out there. I did a search on GitHUB and most of the hits for these method names use the same method signature as Optional – so those might be able to stay. Is it a reckless choice? Yup. But look at the alternative. Every day more and more libraries are having to adopt and adapt to use Optional<T>s (Jackson, Gson, Jersey, Hibernate, Immutables, Lombok, Spring, etc.) and everyday users are introducing more and more fucked-up usages of Optional<T>.

So while there are no perfect solutions there are better solutions than a wrapper class with ambiguous semantics. The presence of Optional<T> behavior on an object should not be optional. Optional behavior should be implicit on every object whether it is null or not. Introducing Optional<T> into Java 8 was a simple solution to a complex problem. But it has introduced its own set problem that are far reaching and will last in perpetuity.

So how will I use Optional<T> now? Until something else comes along I will follow the advice of Brian Goetz.

Simple Basic Authentication for a Jersey JAX-RS Server

If you are working on an MVP for a start-up you are probably doing many things you have never done before. If one of those things is a REST server please make sure AT THE VERY LEAST to require that clients use HTTPS and that you add an API KEY and SECRET for your REST server using Basic Auth.

To learn the basics about basic auth go here or just google around.

For this post I assume you are using some language that can make use of Jersey JAX-RS version 2.7 or higher.

Okay, so you have written a fucking awesome REST server for your mobile apps to use. You are such a bad ass because it is also running on something like heroku. The REST server does what the client needs, the clients are basically done, so you are ready to go live with your MVP. STOP!! Stop what you are doing and ask yourself this. Have you added at least a basic layer of security? If you haven’t you will most definitely regret it later. Once you have released a client without HTTPS and Basic Auth it is hard to go back and require it. That’s because those HTTP clients will still need to be supported even after you introduce a new version that has some security. So stop what you are doing and follow these simple steps.

  1. Go to my GitHUB repository and add this code to your project.
  2. Annotate your resource classes with @BasicAuthentication
  3. Add the BasicAuthenticationFeature to your ResourceConfig.
  4. Distribute your new API KEY and API SECRET to your client to use.
  5. Have the client modify its REST calls to your API to use Basic Auth.
  6. Feel a lot better about everything in the universe.

You don’t really need to read on at this point. The unit tests show you how to annotate some resources and generate keys.  You can always add layers of user specific authentication or authorization on top of this but at least start with Basic Authentication and HTTPS.

If you are going to make your entire REST API authenticated you will only need a ContainerRequestFilter that performs the authentication. This will work in most cases but sometimes you will also have some unauthenticated resources in your API. For example, an HTML resource that you want anyone to be able to hit (not just your authenticated mobile client). For those cases I created an annotation named @BasicAuthentication. If present code will be authenticated. If not present or set to false (e.g. @BasicAuthentication(false)) then the authenticating filter will not be invoked for that resource method.

All the code you need is in GitHUB at Jester.

To use it in your server you need to generate a KEY and SECRET for your API and install the feature.

To generate the API key and secret I just use the result of calling UUID.nextRandom().toString() twice. Once for the KEY and again for the SECRET. Then I create my Jersey ResourceConfig like this:

public class JerseyResourceConfiguration extends ResourceConfig {

    private static final String API_KEY = "8694c2e7-a9c4-4646-acdb-8dbe8e8829af";
    private static final String API_SECRET = "75c4bca4-5bf9-4c91-99cc-e640ee6cded7";

    public JerseyResourceConfiguration() {
        register(new BasicAuthenticationFeature(API_KEY, API_SECRET));
        // whatever else you register

That’s pretty much it. I keep my source and jars in a secure location so I embed the string literals into the source. You may want to take another approach to ensure the privacy of these strings. You should find lots of examples in the test directory of the GitHUB project.

Next give these strings to your client developers and tell them to make REST calls with basic auth. You can also use curl to test your API.

-u "8694c2e7-a9c4-4646-acdb-8dbe8e8829af:75c4bca4-5bf9-4c91-99cc-e640ee6cded7" https://myrestapi/dosomething

I’ll try to expand on all this later but I wanted to get something up quickly for those of you who would otherwise push an unsecured REST API onto Heroku.

Don’t Create a Blob




What is my product’s special purpose?



You have to make choices



You Aren’t Gonna Need It and MVP






10 or so Signs that R&D is Sick




1. Process over Productivity


2. Technology Centric Architecture


3. Managers That Don’t Code Making Technology Decisions


4. Having a “No Heroes” Attitude


5. A Separate Bug Fixing Team




7. The UX Team is Marginalized


8. The Engineering Team has no Customer Contact


9. The Team is Resistant to Feature Changes


10. Tension Between R&D and Product Management






When you’re green you grow when you’re ripe you rot.


Okay, so I’m going to start my blog with this video. Anyone who knows me knows how much I like this talk by Derek Sivers on failure. That he is from Portland is just an added bonus.

Probably my favorite part of this presentation is where he talks about Picasso being video taped while in the process of painting. The painting goes through many revisions, at some points Picasso even seems unsure of himself. Ideas are crossed out and redone until he gets to his finished product. He made many mistakes to get to his completed painting. The big take away here is that you cannot compare your internal self to the external self of someone else. Think about it. You know everything about you, you know all of your insecurities, the circuitous route you take to finally arrive at a decision, etc. For someone else all you know is the final product the present – you do not see their internal process. So do not compare your inside to the outside of someone else. You will always come out short.

Why You Need to Fail – by Derek Sivers from Derek Sivers on Vimeo.