Q:

Spring Security Oauth - Custom format for OAuth2Exceptions

The error format of spring security oauth conforms with the OAuth spec and looks like this.
{
  "error":"insufficient_scope",
  "error_description":"Insufficient scope for this resource",
  "scope":"do.something"
}

Especially on a resource server I find it a bit strange to get a different error format for authentication issues. So I would like to change the way this exception is rendered.
The documentation says

Error handling in an Authorization Server uses standard Spring MVC
  features, namely @ExceptionHandler methods

So I tried something like this to customize the format of the error:
@ControllerAdvice
@Order(Ordered.HIGHEST_PRECEDENCE)
public class MyErrorHandler {

    @ExceptionHandler(value = {InsufficientScopeException.class})
    ResponseEntity<MyErrorRepresentation> handle(RuntimeException ex, HttpServletRequest request) {
        return errorResponse(HttpStatus.FORBIDDEN,
                MyErrorRepresentation.builder()
                        .errorId("insufficient.scope")
                        .build(),
                request);
    }
}

But this does not work. 
Looking at the code, all the error rendering seems to be done in DefaultWebResponseExceptionTranslator#handleOAuth2Exception. But implementing a custom WebResponseExceptionTranslator would not allow changing the format.
Any hints?

A:

First of all，some knowledge for Spring Security OAuth2.

OAuth2 has two main parts

AuthorizationServer : /oauth/token, get token
ResourceServer : url resource priviledge management

Spring Security add filter to the filter chains of server container, so the exception of Spring Security will not reach @ControllerAdvice

Then, custom OAuth2Exceptions should consider for  AuthorizationServer and ResourceServer.
This is configuration
@Configuration
@EnableAuthorizationServer
public class OAuthSecurityConfig extends AuthorizationServerConfigurerAdapter {
    @Override
    public void configure(AuthorizationServerEndpointsConfigurer endpoints) throws Exception {
        //for custom
        endpoints.exceptionTranslator(new MyWebResponseExceptionTranslator());
    }
}

@Configuration
@EnableResourceServer
public  class ResourceServerConfiguration extends ResourceServerConfigurerAdapter {
        @Override
        public void configure(ResourceServerSecurityConfigurer resources) {
            // format message
            resources.authenticationEntryPoint(new MyAuthenticationEntryPoint());
            resources.accessDeniedHandler(new MyAccessDeniedHandler());
        }
}

MyWebResponseExceptionTranslator is translate the exception to ourOAuthException and we custom ourOAuthException serializer by jackson, which way is same by default the OAuth2 use.
@JsonSerialize(using = OAuth2ExceptionJackson1Serializer.class)
public class OAuth2Exception extends RuntimeException {

other custom handle class stuff
import org.springframework.http.HttpStatus;
import org.springframework.http.ResponseEntity;
import org.springframework.security.core.AuthenticationException;
import org.springframework.security.oauth2.common.exceptions.OAuth2Exception;
import org.springframework.security.oauth2.provider.error.WebResponseExceptionTranslator;

/**
 * @author qianggetaba
 * @date 2019/6/21
 */
public class MyWebResponseExceptionTranslator implements WebResponseExceptionTranslator {
    @Override
    public ResponseEntity<OAuth2Exception> translate(Exception exception) throws Exception {
        if (exception instanceof OAuth2Exception) {
            OAuth2Exception oAuth2Exception = (OAuth2Exception) exception;
            return ResponseEntity
                    .status(oAuth2Exception.getHttpErrorCode())
                    .body(new CustomOauthException(oAuth2Exception.getMessage()));
        }else if(exception instanceof AuthenticationException){
            AuthenticationException authenticationException = (AuthenticationException) exception;
            return ResponseEntity
                    .status(HttpStatus.UNAUTHORIZED)
                    .body(new CustomOauthException(authenticationException.getMessage()));
        }
        return ResponseEntity
                .status(HttpStatus.OK)
                .body(new CustomOauthException(exception.getMessage()));
    }
}

import com.fasterxml.jackson.databind.annotation.JsonSerialize;
import org.springframework.security.oauth2.common.exceptions.OAuth2Exception;

/**
 * @author qianggetaba
 * @date 2019/6/21
 */
@JsonSerialize(using = CustomOauthExceptionSerializer.class)
public class CustomOauthException extends OAuth2Exception {
    public CustomOauthException(String msg) {
        super(msg);
    }
}

import com.fasterxml.jackson.core.JsonGenerator;
import com.fasterxml.jackson.databind.SerializerProvider;
import com.fasterxml.jackson.databind.ser.std.StdSerializer;

import java.io.IOException;
import java.util.Arrays;
import java.util.Map;

/**
 * @author qianggetaba
 * @date 2019/6/21
 */
public class CustomOauthExceptionSerializer extends StdSerializer<CustomOauthException> {

    public CustomOauthExceptionSerializer() {
        super(CustomOauthException.class);
    }

    @Override
    public void serialize(CustomOauthException value, JsonGenerator jsonGenerator, SerializerProvider serializerProvider) throws IOException {
        jsonGenerator.writeStartObject();
        jsonGenerator.writeNumberField("code4444", value.getHttpErrorCode());
        jsonGenerator.writeBooleanField("status", false);
        jsonGenerator.writeObjectField("data", null);
        jsonGenerator.writeObjectField("errors", Arrays.asList(value.getOAuth2ErrorCode(),value.getMessage()));
        if (value.getAdditionalInformation()!=null) {
            for (Map.Entry<String, String> entry : value.getAdditionalInformation().entrySet()) {
                String key = entry.getKey();
                String add = entry.getValue();
                jsonGenerator.writeStringField(key, add);
            }
        }
        jsonGenerator.writeEndObject();
    }
}

for custom ResourceServer exception
import com.fasterxml.jackson.databind.ObjectMapper;
import org.springframework.security.core.AuthenticationException;
import org.springframework.security.web.AuthenticationEntryPoint;

import javax.servlet.ServletException;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import java.util.Date;
import java.util.HashMap;
import java.util.Map;

/**
 * @author qianggetaba
 * @date 2019/6/21
 */
public class MyAuthenticationEntryPoint implements AuthenticationEntryPoint {
    @Override
    public void commence(HttpServletRequest request, HttpServletResponse response,
                         AuthenticationException authException)
            throws ServletException {

        Map map = new HashMap();
        map.put("errorentry", "401");
        map.put("message", authException.getMessage());
        map.put("path", request.getServletPath());
        map.put("timestamp", String.valueOf(new Date().getTime()));
        response.setContentType("application/json");
        response.setStatus(HttpServletResponse.SC_UNAUTHORIZED);
        try {
            ObjectMapper mapper = new ObjectMapper();
            mapper.writeValue(response.getOutputStream(), map);
        } catch (Exception e) {
            throw new ServletException();
        }
    }

}

import com.fasterxml.jackson.databind.ObjectMapper;
import org.springframework.security.access.AccessDeniedException;
import org.springframework.security.web.access.AccessDeniedHandler;

import javax.servlet.ServletException;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import java.io.IOException;
import java.util.Date;
import java.util.HashMap;
import java.util.Map;

/**
 * @author qianggetaba
 * @date 2019/6/21
 */
public class MyAccessDeniedHandler implements AccessDeniedHandler{
    @Override
    public void handle(HttpServletRequest request, HttpServletResponse response, AccessDeniedException accessDeniedException) throws IOException, ServletException {
        response.setContentType("application/json;charset=UTF-8");
        Map map = new HashMap();
        map.put("errorauth", "400");
        map.put("message", accessDeniedException.getMessage());
        map.put("path", request.getServletPath());
        map.put("timestamp", String.valueOf(new Date().getTime()));
        response.setContentType("application/json");
        response.setStatus(HttpServletResponse.SC_UNAUTHORIZED);
        try {
            ObjectMapper mapper = new ObjectMapper();
            mapper.writeValue(response.getOutputStream(), map);
        } catch (Exception e) {
            throw new ServletException();
        }
    }
}

A:

I found a similar question with answers that really helped my solving this - Handle spring security authentication exceptions with @ExceptionHandler
But my question is specifically about spring-security-oauth2 - so I think it is still worth stating the answer specific to spring-security-oauth2. My solution was picked from different answers to the question mentioned above.
My samples work for spring-security-oauth2 2.0.13
So the solution for me to achieve a different custom error structure for oauth2 errors on resource server resources was to register a custom OAuth2AuthenticationEntryPoint and OAuth2AccessDeniedHandler that I register using a ResourceServerConfigurerAdapter. It is worth mentioning that this is only changing the format for ResourceServer endpoints - and not the AuthorizationServer endpoints like the TokenEndpoint.
class MyCustomOauthErrorConversionConfigurerAdapter extends ResourceServerConfigurerAdapter {

  @Override
  public void configure(ResourceServerSecurityConfigurer configurer) throws Exception {
    configurer.authenticationEntryPoint(new MyCustomOauthErrorOAuth2AuthenticationEntryPoint());
    configurer.accessDeniedHandler(new MyCustomOauthErrorOAuth2AccessDeniedHandler());
  }
}

I could not reuse the functionality in OAuth2AuthenticationEntryPoint and OAuth2AccessDeniedHandler because the relevant methods translate the exception and flush it in the same method. So I needed to copy some code:
        public class MyCustomOauthErrorOAuth2AccessDeniedHandler extends OAuth2AccessDeniedHandler {

  private final MyCustomOauthErrorOAuth2SecurityExceptionHandler oAuth2SecurityExceptionHandler = new MyCustomOauthErrorOAuth2SecurityExceptionHandler();

  /**
  * Does exactly what OAuth2AccessDeniedHandler does only that the body is transformed to {@link MyCustomOauthError} before rendering the exception
  */
  @Override
  public void handle(HttpServletRequest request, HttpServletResponse response, org.springframework.security.access.AccessDeniedException authException)
  throws IOException, ServletException {
    oAuth2SecurityExceptionHandler.handle(request, response, authException, this::enhanceResponse);
  }
}

public class ExceptionMessageOAuth2AuthenticationEntryPoint extends OAuth2AuthenticationEntryPoint {

  private final MyCustomOauthErrorOAuth2SecurityExceptionHandler oAuth2SecurityExceptionHandler = new MyCustomOauthErrorOAuth2SecurityExceptionHandler();

  /**
  * Does exactly what OAuth2AuthenticationEntryPoint does only that the body is transformed to {@link MyCustomOauthError} before rendering the exception
  */
  @Override
  public void commence(HttpServletRequest request, HttpServletResponse response, AuthenticationException authException) throws IOException, ServletException {
    oAuth2SecurityExceptionHandler.handle(request, response, authException, this::enhanceResponse);
  }
}

@RequiredArgsConstructor
public class MyCustomOauthErrorOAuth2SecurityExceptionHandler {

  private final WebResponseExceptionTranslator exceptionTranslator = new DefaultWebResponseExceptionTranslator();

  private final OAuth2ExceptionRenderer exceptionRenderer = new DefaultOAuth2ExceptionRenderer();

  private final HandlerExceptionResolver handlerExceptionResolver = new DefaultHandlerExceptionResolver();

  /**
  * This is basically what {@link org.springframework.security.oauth2.provider.error.AbstractOAuth2SecurityExceptionHandler#doHandle(HttpServletRequest, HttpServletResponse, Exception)} does.
  */
  public void handle(HttpServletRequest request, HttpServletResponse response, RuntimeException authException,
    BiFunction<ResponseEntity<OAuth2Exception>, Exception, ResponseEntity<OAuth2Exception>> oauthExceptionEnhancer)
    throws IOException, ServletException {

      try {
        ResponseEntity<OAuth2Exception> defaultErrorResponse = exceptionTranslator.translate(authException);
        defaultErrorResponse = oauthExceptionEnhancer.apply(defaultErrorResponse, authException);

        //this is the actual translation of the error 
        final MyCustomOauthError customErrorPayload = 
        MyCustomOauthError.builder()
        .errorId(defaultErrorResponse.getBody().getOAuth2ErrorCode())
        .message(defaultErrorResponse.getBody().getMessage())
        .details(defaultErrorResponse.getBody().getAdditionalInformation() == null ? emptyMap() : defaultErrorResponse.getBody().getAdditionalInformation())
        .build();

        final ResponseEntity<MyCustomOauthError> responseEntity = new ResponseEntity<>(customErrorPayload, defaultErrorResponse.getHeaders(), defaultErrorResponse.getStatusCode());

        exceptionRenderer.handleHttpEntityResponse(responseEntity, new ServletWebRequest(request, response));
        response.flushBuffer();
      } catch (ServletException e) {
        // Re-use some of the default Spring dispatcher behaviour - the exception came from the filter chain and
        // not from an MVC handler so it won't be caught by the dispatcher (even if there is one)
        if (handlerExceptionResolver.resolveException(request, response, this, e) == null) {
          throw e;
        }
      } catch (IOException | RuntimeException e) {
        throw e;
      } catch (Exception e) {
        // Wrap other Exceptions. These are not expected to happen
        throw new RuntimeException(e);
      }
  }
}

