You will be provided with a partial code base and an issue statement explaining a problem to resolve.
<issue>
Wrong hash value for 2020-06-23 and 2020-06-24
<!--
Thanks for proposing an enhancement 🙌 ❤️

Before opening a new issue, please make sure that we do not have any duplicates already open. You can ensure this by searching the issue list for this repository. If there is a duplicate, please close your issue and add a comment to the existing issue instead.
-->


## Current Implementation
<!-- Describe or point to the current implementation that you would like to see improved -->
Old keys are deleted and the remaining keys are repacked by day or by hour.
60 keys moved from 24.06.2020 to 23.06.2020.
44 keys are deleted from 23.06.2020
The hour files 2020-06-23-hour-08.zip, 2020-06-23-hour-13.zip, 2020-06-23-hour-17.zip are repacked to
2020-06-23-hour-10.zip, 2020-06-23-hour-15.zip, 2020-06-23-hour-18.zip
(Even the old hour files are superseded they are still on the server.)
## Suggested Enhancement
<!-- Outline the idea of your enhancement, by e.g., describing the algorithm you propose. You can also create a Pull Request to outline your idea -->
Leave the files as they are, do not remove old keys. 
## Expected Benefits
<!-- Summarize how your enhancement could aid the implementation (performance, readability, memory consumption, battery consumption, etc.). Please also back up with measurements or give detailed explanations for reduced runtimes, memory consumption, etc.  -->
The files downloaded by the app have the same hash value as the file on the server.

</issue>

<code>
[start of README.md]
1 <!-- markdownlint-disable MD041 -->
2 <h1 align="center">
3     Corona-Warn-App Server
4 </h1>
5 
6 <p align="center">
7     <a href="https://github.com/corona-warn-app/cwa-server/commits/" title="Last Commit"><img src="https://img.shields.io/github/last-commit/corona-warn-app/cwa-server?style=flat"></a>
8     <a href="https://github.com/corona-warn-app/cwa-server/issues" title="Open Issues"><img src="https://img.shields.io/github/issues/corona-warn-app/cwa-server?style=flat"></a>
9     <a href="https://circleci.com/gh/corona-warn-app/cwa-server" title="Build Status"><img src="https://circleci.com/gh/corona-warn-app/cwa-server.svg?style=shield&circle-token=4ab059989d10709df19eb4b98ab7c121a25e981a"></a>
10         <a href="https://sonarcloud.io/dashboard?id=corona-warn-app_cwa-server" title="Quality Gate"><img src="https://sonarcloud.io/api/project_badges/measure?project=corona-warn-app_cwa-server&metric=alert_status"></a>
11         <a href="https://sonarcloud.io/component_measures?id=corona-warn-app_cwa-server&metric=Coverage&view=list" title="Coverage"><img src="https://sonarcloud.io/api/project_badges/measure?project=corona-warn-app_cwa-server&metric=coverage"></a>
12     <a href="https://github.com/corona-warn-app/cwa-server/blob/master/LICENSE" title="License"><img src="https://img.shields.io/badge/License-Apache%202.0-green.svg?style=flat"></a>
13 </p>
14 
15 <p align="center">
16   <a href="#development">Development</a> •
17   <a href="#service-apis">Service APIs</a> •
18   <a href="#documentation">Documentation</a> •
19   <a href="#support-and-feedback">Support</a> •
20   <a href="#how-to-contribute">Contribute</a> •
21   <a href="#contributors">Contributors</a> •
22   <a href="#repositories">Repositories</a> •
23   <a href="#licensing">Licensing</a>
24 </p>
25 
26 The goal of this project is to develop the official Corona-Warn-App for Germany based on the exposure notification API from [Apple](https://www.apple.com/covid19/contacttracing/) and [Google](https://www.google.com/covid19/exposurenotifications/). The apps (for both iOS and Android) use Bluetooth technology to exchange anonymous encrypted data with other mobile phones (on which the app is also installed) in the vicinity of an app user's phone. The data is stored locally on each user's device, preventing authorities or other parties from accessing or controlling the data. This repository contains the **implementation of the server for encryption keys** for the Corona-Warn-App.
27 
28 In this documentation, Corona-Warn-App services are also referred to as CWA services.
29 
30 ## Architecture Overview
31 
32 You can find the architecture overview [here](/docs/ARCHITECTURE.md), which will give you
33 a good starting point in how the backend services interact with other services, and what purpose
34 they serve.
35 
36 ## Development
37 
38 After you've checked out this repository, you can run the application in one of the following ways:
39 
40 * As a [Docker](https://www.docker.com/)-based deployment on your local machine. You can run either:
41   * Single components using the respective Dockerfile or
42   * The full backend using the Docker Compose (which is considered the most convenient way)
43 * As a [Maven](https://maven.apache.org)-based build on your local machine.
44   If you want to develop something in a single component, this approach is preferable.
45 
46 ### Docker-Based Deployment
47 
48 If you want to use Docker-based deployment, you need to install Docker on your local machine. For more information about downloading and installing Docker, see the [official Docker documentation](https://docs.docker.com/get-docker/).
49 
50 #### Running the Full CWA Backend Using Docker Compose
51 
52 For your convenience, a full setup for local development and testing purposes, including the generation of test data has been prepared using [Docker Compose](https://docs.docker.com/compose/reference/overview/). To build the backend services, run ```docker-compose build``` in the repository's root directory. A default configuration file can be found under ```.env``` in the root folder of the repository. If the endpoints are to be exposed to the network the default values in this file should be changed before docker-compose is run.
53 
54 Once the services are built, you can start the whole backend using ```docker-compose up```.
55 The distribution service runs once and then finishes. If you want to trigger additional distribution runs, run ```docker-compose run distribution```.
56 
57 The docker-compose contains the following services:
58 
59 Service           | Description | Endpoint and Default Credentials
60 ------------------|-------------|-----------
61 submission        | The Corona-Warn-App submission service                                                      | `http://localhost:8000` <br> `http://localhost:8006` (for actuator endpoint)
62 distribution      | The Corona-Warn-App distribution service                                                    | NO ENDPOINT
63 postgres          | A [postgres] database installation                                                          | `localhost:8001` <br> `postgres:5432` (from containerized pgadmin) <br> Username: postgres <br> Password: postgres
64 pgadmin           | A [pgadmin](https://www.pgadmin.org/) installation for the postgres database                | `http://localhost:8002` <br> Username: user@domain.com <br> Password: password
65 cloudserver       | [Zenko CloudServer] is a S3-compliant object store  | `http://localhost:8003/` <br> Access key: accessKey1 <br> Secret key: verySecretKey1
66 verification-fake | A very simple fake implementation for the tan verification.                                 | `http://localhost:8004/version/v1/tan/verify` <br> The only valid tan is `edc07f08-a1aa-11ea-bb37-0242ac130002`.
67 
68 ##### Known Limitation
69 
70 In rare cases the docker-compose runs into a timing issue if the distribution service starts before the bucket of the objectstore was created. This is not a big issue as you can simply run ```docker-compose run distribution``` to trigger additional distribution runs after the objectstore was initialized.
71 
72 #### Running Single CWA Services Using Docker
73 
74 If you would like to build and run a single CWA service, it's considered easiest to run them in a Docker environment. You can do this using the script provided in the respective CWA service directory. The Docker script first builds the CWA service and then creates an image for the runtime, which means that there are no additional dependencies for you to install.
75 
76 To build and run the distribution service, run the following command:
77 
78 ```bash
79 ./services/distribution/build_and_run.sh
80 ```
81 
82 To build and run the submission service, run the following command:
83 
84 ```bash
85 ./services/submission/build_and_run.sh
86 ```
87 
88 The submission service is available on [localhost:8080](http://localhost:8080 ).
89 
90 ### Maven-Based Build
91 
92 If you want to actively develop in one of the CWA services, the Maven-based runtime is most suitable.
93 To prepare your machine to run the CWA project locally, we recommend that you first ensure that you've installed the following:
94 
95 * Minimum JDK Version 11: [OpenJDK](https://openjdk.java.net/) / [SapMachine](https://sap.github.io/SapMachine/)
96 * [Maven 3.6](https://maven.apache.org/)
97 * [Postgres]
98 * [Zenko CloudServer]
99 
100 If you are already running a local Postgres, you need to create a database `cwa` and run the following setup scripts:
101 
102 * Create the different CWA roles first by executing [create-roles.sql](setup/create-roles.sql).
103 * Create local database users for the specific roles by running [create-users.sql](./local-setup/create-users.sql).
104 * It is recommended to also run [enable-test-data-docker-compose.sql](./local-setup/enable-test-data-docker-compose.sql)
105 , which enables the test data generation profile. If you already had CWA running before and an existing `diagnosis-key`
106 table on your database, you need to run [enable-test-data.sql](./local-setup/enable-test-data.sql) instead.
107 
108 You can also use `docker-compose` to start Postgres and Zenko. If you do that, you have to
109 set the following environment-variables when running the Spring project:
110 
111 For the distribution module:
112 
113 ```bash
114 POSTGRESQL_SERVICE_PORT=8001
115 VAULT_FILESIGNING_SECRET=</path/to/your/private_key>
116 SPRING_PROFILES_ACTIVE=signature-dev,disable-ssl-client-postgres
117 ```
118 
119 For the submission module:
120 
121 ```bash
122 POSTGRESQL_SERVICE_PORT=8001
123 SPRING_PROFILES_ACTIVE=disable-ssl-server,disable-ssl-client-postgres,disable-ssl-client-verification,disable-ssl-client-verification-verify-hostname
124 ```
125 
126 #### Configure
127 
128 After you made sure that the specified dependencies are running, configure them in the respective configuration files.
129 
130 * Configure the Postgres connection in the [submission config](./services/submission/src/main/resources/application.yaml) and in the [distribution config](./services/distribution/src/main/resources/application.yaml)
131 * Configure the S3 compatible object storage in the [distribution config](./services/distribution/src/main/resources/application.yaml)
132 * Configure the private key for the distribution service, the path need to be prefixed with `file:`
133   * `VAULT_FILESIGNING_SECRET` should be the path to the private key, example available in `<repo-root>/docker-compose-test-secrets/private.pem`
134 
135 #### Build
136 
137 After you've checked out the repository, to build the project, run ```mvn install``` in your base directory.
138 
139 #### Run
140 
141 Navigate to the service you want to start and run the spring-boot:run target. The configured Postgres and the configured S3 compliant object storage are used as default. When you start the submission service, the endpoint is available on your local port 8080.
142 
143 If you want to start the submission service, for example, you start it as follows:
144 
145 ```bash
146   cd services/submission/
147   mvn spring-boot:run
148 ```
149 
150 #### Debugging
151 
152 To enable the `DEBUG` log level, you can run the application using the Spring `dev` profile.
153 
154 ```bash
155 mvn spring-boot:run -Dspring.profiles.active=dev
156 ```
157 
158 To be able to set breakpoints (e.g. in IntelliJ), it may be necessary to use the ```-Dspring-boot.run.fork=false``` parameter.
159 
160 ## Service APIs
161 
162 The API that is being exposed by the backend services is documented in an [OpenAPI](https://www.openapis.org/) specification. The specification files are available at the following locations:
163 
164 Service                   | OpenAPI Specification
165 --------------------------|-------------
166 Submission Service        | [services/submission/api_v1.json](https://github.com/corona-warn-app/cwa-server/raw/master/services/submission/api_v1.json)
167 Distribution Service      | [services/distribution/api_v1.json](https://github.com/corona-warn-app/cwa-server/raw/master/services/distribution/api_v1.json)
168 
169 ## Spring Profiles
170 
171 ### Distribution
172 
173 See [Distribution Service - Spring Profiles](/docs/DISTRIBUTION.md#spring-profiles).
174 
175 ### Submission
176 
177 See [Submission Service - Spring Profiles](/docs/SUBMISSION.md#spring-profiles).
178 
179 ## Documentation
180 
181 The full documentation for the Corona-Warn-App can be found in the [cwa-documentation](https://github.com/corona-warn-app/cwa-documentation) repository. The documentation repository contains technical documents, architecture information, and whitepapers related to this implementation.
182 
183 The documentation for cwa-server can be found under the [/docs](./docs) folder.
184 
185 The JavaDoc documentation for cwa-server is hosted by Github Pages at [https://corona-warn-app.github.io/cwa-server](https://corona-warn-app.github.io/cwa-server).
186 
187 ## Support and Feedback
188 
189 The following channels are available for discussions, feedback, and support requests:
190 
191 | Type                     | Channel                                                |
192 | ------------------------ | ------------------------------------------------------ |
193 | **General Discussion**   | <a href="https://github.com/corona-warn-app/cwa-documentation/issues/new/choose" title="General Discussion"><img src="https://img.shields.io/github/issues/corona-warn-app/cwa-documentation/question.svg?style=flat-square"></a> </a>   |
194 | **Concept Feedback**    | <a href="https://github.com/corona-warn-app/cwa-documentation/issues/new/choose" title="Open Concept Feedback"><img src="https://img.shields.io/github/issues/corona-warn-app/cwa-documentation/architecture.svg?style=flat-square"></a>  |
195 | **Backend Issue**    | <a href="https://github.com/corona-warn-app/cwa-server/issues/new/choose" title="Open Backend Issue"><img src="https://img.shields.io/github/issues/corona-warn-app/cwa-server?style=flat-square"></a>  |
196 | **Other Requests**    | <a href="mailto:corona-warn-app.opensource@sap.com" title="Email CWA Team"><img src="https://img.shields.io/badge/email-CWA%20team-green?logo=mail.ru&style=flat-square&logoColor=white"></a>   |
197 
198 ## How to Contribute
199 
200 Contribution and feedback are encouraged and always welcome. For more information about how to contribute, the project structure, as well as additional contribution information, see our [Contribution Guidelines](./CONTRIBUTING.md). By participating in this project, you agree to abide by its [Code of Conduct](./CODE_OF_CONDUCT.md) at all times.
201 
202 ## Contributors
203 
204 The German government has asked SAP and Deutsche Telekom to develop the Corona-Warn-App for Germany as open source software. Deutsche Telekom is providing the network and mobile technology and will operate and run the backend for the app in a safe, scalable and stable manner. SAP is responsible for the app development, its framework and the underlying platform. Therefore, development teams of SAP and Deutsche Telekom are contributing to this project. At the same time our commitment to open source means that we are enabling -in fact encouraging- all interested parties to contribute and become part of its developer community.
205 
206 ## Repositories
207 
208 The following public repositories are currently available for the Corona-Warn-App:
209 
210 | Repository          | Description                                                           |
211 | ------------------- | --------------------------------------------------------------------- |
212 | [cwa-documentation] | Project overview, general documentation, and white papers            |
213 | [cwa-server]        | Backend implementation for the Apple/Google exposure notification API|
214 | [cwa-verification-server] | Backend implementation of the verification process|
215 
216 [cwa-documentation]: https://github.com/corona-warn-app/cwa-documentation
217 [cwa-server]: https://github.com/corona-warn-app/cwa-server
218 [cwa-verification-server]: https://github.com/corona-warn-app/cwa-verification-server
219 [Postgres]: https://www.postgresql.org/
220 [HSQLDB]: http://hsqldb.org/
221 [Zenko CloudServer]: https://github.com/scality/cloudserver
222 
223 ## Licensing
224 
225 Copyright (c) 2020 SAP SE or an SAP affiliate company.
226 
227 Licensed under the **Apache License, Version 2.0** (the "License"); you may not use this file except in compliance with the License.
228 
229 You may obtain a copy of the License at <https://www.apache.org/licenses/LICENSE-2.0>.
230 
231 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the [LICENSE](./LICENSE) for the specific language governing permissions and limitations under the License.
232 
[end of README.md]
[start of docker-compose.yml]
1 version: '3'
2 services:
3   submission:
4     build:
5       context: ./
6       dockerfile: ./services/submission/Dockerfile
7     depends_on:
8       - postgres
9       - verification-fake
10     ports:
11       - "8000:8080"
12       - "8006:8081"
13     environment:
14       SPRING_PROFILES_ACTIVE: dev,disable-ssl-server,disable-ssl-client-postgres,disable-ssl-client-verification,disable-ssl-client-verification-verify-hostname
15       POSTGRESQL_SERVICE_PORT: '5432'
16       POSTGRESQL_SERVICE_HOST: postgres
17       POSTGRESQL_DATABASE: ${POSTGRES_DB}
18       POSTGRESQL_PASSWORD_SUBMISION: ${POSTGRES_SUBMISSION_PASSWORD}
19       POSTGRESQL_USER_SUBMISION: ${POSTGRES_SUBMISSION_USER}
20       POSTGRESQL_PASSWORD_FLYWAY: ${POSTGRES_FLYWAY_PASSWORD}
21       POSTGRESQL_USER_FLYWAY: ${POSTGRES_FLYWAY_USER}
22       VERIFICATION_BASE_URL: http://verification-fake:8004
23   distribution:
24     build:
25       context: ./
26       dockerfile: ./services/distribution/Dockerfile
27     depends_on:
28      - postgres
29      - objectstore
30      - create-bucket
31     environment:
32       SPRING_PROFILES_ACTIVE: dev,signature-dev,testdata,disable-ssl-client-postgres
33       POSTGRESQL_SERVICE_PORT: '5432'
34       POSTGRESQL_SERVICE_HOST: postgres
35       POSTGRESQL_DATABASE: ${POSTGRES_DB}
36       POSTGRESQL_PASSWORD_DISTRIBUTION: ${POSTGRES_DISTRIBUTION_PASSWORD}
37       POSTGRESQL_USER_DISTRIBUTION: ${POSTGRES_DISTRIBUTION_USER}
38       POSTGRESQL_PASSWORD_FLYWAY: ${POSTGRES_FLYWAY_PASSWORD}
39       POSTGRESQL_USER_FLYWAY: ${POSTGRES_FLYWAY_USER}
40       # Settings for the S3 compatible objectstore
41       CWA_OBJECTSTORE_ACCESSKEY: ${OBJECTSTORE_ACCESSKEY}
42       CWA_OBJECTSTORE_SECRETKEY: ${OBJECTSTORE_SECRETKEY}
43       CWA_OBJECTSTORE_ENDPOINT: http://objectstore
44       CWA_OBJECTSTORE_BUCKET: cwa
45       CWA_OBJECTSTORE_PORT: 8000
46       services.distribution.paths.output: /tmp/distribution
47       # Settings for cryptographic artifacts
48       VAULT_FILESIGNING_SECRET: ${SECRET_PRIVATE}
49     volumes:
50       - ./docker-compose-test-secrets:/secrets
51   postgres:
52     image: postgres:11.8
53     restart: always
54     ports:
55       - "8001:5432"
56     environment:
57       PGDATA: /data/postgres
58       POSTGRES_DB: ${POSTGRES_DB}
59       POSTGRES_USER: ${POSTGRES_USER}
60       POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
61     volumes:
62        - postgres_volume:/data/postgres
63        - ./setup/setup-roles.sql:/docker-entrypoint-initdb.d/1-roles.sql
64        - ./local-setup/create-users.sql:/docker-entrypoint-initdb.d/2-users.sql
65        - ./local-setup/enable-test-data-docker-compose.sql:/docker-entrypoint-initdb.d/3-enable-testdata.sql
66   pgadmin:
67     container_name: pgadmin_container
68     image: dpage/pgadmin4
69     volumes:
70        - pgadmin_volume:/root/.pgadmin
71     ports:
72       - "8002:80"
73     restart: unless-stopped
74     depends_on:
75       - postgres
76     environment:
77       PGADMIN_DEFAULT_EMAIL: ${PGADMIN_DEFAULT_EMAIL}
78       PGADMIN_DEFAULT_PASSWORD: ${PGADMIN_DEFAULT_PASSWORD}
79   objectstore:
80     image: "zenko/cloudserver"
81     volumes:
82       - objectstore_volume:/data
83     ports:
84       - "8003:8000"
85     environment:
86       ENDPOINT: objectstore
87       REMOTE_MANAGEMENT_DISABLE: 1
88       SCALITY_ACCESS_KEY_ID: ${OBJECTSTORE_ACCESSKEY}
89       SCALITY_SECRET_ACCESS_KEY: ${OBJECTSTORE_SECRETKEY}
90   create-bucket:
91     image: amazon/aws-cli
92     environment:
93       - AWS_ACCESS_KEY_ID=${OBJECTSTORE_ACCESSKEY}
94       - AWS_SECRET_ACCESS_KEY=${OBJECTSTORE_SECRETKEY}
95     entrypoint: ["/root/scripts/wait-for-it/wait-for-it.sh", "objectstore:8000", "-t", "30", "--"]
96     volumes:
97       - ./scripts/wait-for-it:/root/scripts/wait-for-it
98     command: aws s3api create-bucket --bucket cwa --endpoint-url http://objectstore:8000 --acl public-read
99     depends_on:
100       - objectstore
101   verification-fake:
102     image: lilienthal/cwa-verification-fake:0.0.3
103     restart: always
104     ports:
105       - "8004:8004"
106 volumes:
107   postgres_volume:
108   pgadmin_volume:
109   objectstore_volume:
110 
[end of docker-compose.yml]
[start of services/distribution/src/main/java/app/coronawarn/server/services/distribution/assembly/diagnosiskeys/ProdDiagnosisKeyBundler.java]
1 /*-
2  * ---license-start
3  * Corona-Warn-App
4  * ---
5  * Copyright (C) 2020 SAP SE and all other contributors
6  * ---
7  * Licensed under the Apache License, Version 2.0 (the "License");
8  * you may not use this file except in compliance with the License.
9  * You may obtain a copy of the License at
10  *
11  *      http://www.apache.org/licenses/LICENSE-2.0
12  *
13  * Unless required by applicable law or agreed to in writing, software
14  * distributed under the License is distributed on an "AS IS" BASIS,
15  * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
16  * See the License for the specific language governing permissions and
17  * limitations under the License.
18  * ---license-end
19  */
20 
21 package app.coronawarn.server.services.distribution.assembly.diagnosiskeys;
22 
23 import static java.time.ZoneOffset.UTC;
24 import static java.util.Collections.emptyList;
25 import static java.util.stream.Collectors.groupingBy;
26 
27 import app.coronawarn.server.common.persistence.domain.DiagnosisKey;
28 import app.coronawarn.server.services.distribution.config.DistributionServiceConfig;
29 import java.time.Duration;
30 import java.time.LocalDateTime;
31 import java.time.temporal.ChronoUnit;
32 import java.util.ArrayList;
33 import java.util.Collection;
34 import java.util.HashMap;
35 import java.util.List;
36 import java.util.Map;
37 import java.util.Optional;
38 import java.util.stream.LongStream;
39 import org.springframework.context.annotation.Profile;
40 import org.springframework.stereotype.Component;
41 
42 /**
43  * An instance of this class contains a collection of {@link DiagnosisKey DiagnosisKeys}, that will be distributed while
44  * respecting expiry policy (keys must be expired for a configurable amount of time before distribution) and shifting
45  * policy (there must be at least a configurable number of keys in a distribution). The policies are configurable
46  * through the properties {@code expiry-policy-minutes} and {@code shifting-policy-threshold}.
47  */
48 @Profile("!demo")
49 @Component
50 public class ProdDiagnosisKeyBundler extends DiagnosisKeyBundler {
51 
52   /**
53    * Creates a new {@link ProdDiagnosisKeyBundler}.
54    */
55   public ProdDiagnosisKeyBundler(DistributionServiceConfig distributionServiceConfig) {
56     super(distributionServiceConfig);
57   }
58 
59   /**
60    * Initializes the internal {@code distributableDiagnosisKeys} map, grouping the diagnosis keys by the date on which
61    * they may be distributed, while respecting the expiry and shifting policies.
62    */
63   @Override
64   protected void createDiagnosisKeyDistributionMap(Collection<DiagnosisKey> diagnosisKeys) {
65     this.distributableDiagnosisKeys.clear();
66     if (diagnosisKeys.isEmpty()) {
67       return;
68     }
69     Map<LocalDateTime, List<DiagnosisKey>> distributableDiagnosisKeysGroupedByExpiryPolicy = new HashMap<>(
70         diagnosisKeys.stream().collect(groupingBy(this::getDistributionDateTimeByExpiryPolicy)));
71     LocalDateTime earliestDistributableTimestamp =
72         getEarliestDistributableTimestamp(distributableDiagnosisKeysGroupedByExpiryPolicy).orElseThrow();
73     LocalDateTime latestDistributableTimestamp = this.distributionTime;
74 
75     List<DiagnosisKey> diagnosisKeyAccumulator = new ArrayList<>();
76     LongStream.range(0, earliestDistributableTimestamp.until(latestDistributableTimestamp, ChronoUnit.HOURS))
77         .forEach(hourCounter -> {
78           LocalDateTime currentHour = earliestDistributableTimestamp.plusHours(hourCounter);
79           Collection<DiagnosisKey> currentHourDiagnosisKeys = Optional
80               .ofNullable(distributableDiagnosisKeysGroupedByExpiryPolicy.get(currentHour))
81               .orElse(emptyList());
82           diagnosisKeyAccumulator.addAll(currentHourDiagnosisKeys);
83           if (diagnosisKeyAccumulator.size() >= minNumberOfKeysPerBundle) {
84             this.distributableDiagnosisKeys.put(currentHour, new ArrayList<>(diagnosisKeyAccumulator));
85             diagnosisKeyAccumulator.clear();
86           }
87         });
88   }
89 
90   private static Optional<LocalDateTime> getEarliestDistributableTimestamp(
91       Map<LocalDateTime, List<DiagnosisKey>> distributableDiagnosisKeys) {
92     return distributableDiagnosisKeys.keySet().stream().min(LocalDateTime::compareTo);
93   }
94 
95   /**
96    * Returns the end of the rolling time window that a {@link DiagnosisKey} was active for as a {@link LocalDateTime}.
97    */
98   private LocalDateTime getExpiryDateTime(DiagnosisKey diagnosisKey) {
99     return LocalDateTime
100         .ofEpochSecond(diagnosisKey.getRollingStartIntervalNumber() * TEN_MINUTES_INTERVAL_SECONDS, 0, UTC)
101         .plusMinutes(diagnosisKey.getRollingPeriod() * 10L);
102   }
103 
104   /**
105    * Calculates the earliest point in time at which the specified {@link DiagnosisKey} can be distributed, while
106    * respecting the expiry policy and the submission timestamp. Before keys are allowed to be distributed, they must be
107    * expired for a configured amount of time.
108    *
109    * @return {@link LocalDateTime} at which the specified {@link DiagnosisKey} can be distributed.
110    */
111   private LocalDateTime getDistributionDateTimeByExpiryPolicy(DiagnosisKey diagnosisKey) {
112     LocalDateTime submissionDateTime = getSubmissionDateTime(diagnosisKey);
113     LocalDateTime expiryDateTime = getExpiryDateTime(diagnosisKey);
114     long minutesBetweenExpiryAndSubmission = Duration.between(expiryDateTime, submissionDateTime).toMinutes();
115     if (minutesBetweenExpiryAndSubmission <= expiryPolicyMinutes) {
116       // truncatedTo floors the value, so we need to add an hour to the DISTRIBUTION_PADDING to compensate that.
117       return expiryDateTime.plusMinutes(expiryPolicyMinutes + 60).truncatedTo(ChronoUnit.HOURS);
118     } else {
119       return submissionDateTime;
120     }
121   }
122 }
123 
[end of services/distribution/src/main/java/app/coronawarn/server/services/distribution/assembly/diagnosiskeys/ProdDiagnosisKeyBundler.java]
[start of services/distribution/src/main/java/app/coronawarn/server/services/distribution/config/DistributionServiceConfig.java]
1 /*-
2  * ---license-start
3  * Corona-Warn-App
4  * ---
5  * Copyright (C) 2020 SAP SE and all other contributors
6  * ---
7  * Licensed under the Apache License, Version 2.0 (the "License");
8  * you may not use this file except in compliance with the License.
9  * You may obtain a copy of the License at
10  *
11  *      http://www.apache.org/licenses/LICENSE-2.0
12  *
13  * Unless required by applicable law or agreed to in writing, software
14  * distributed under the License is distributed on an "AS IS" BASIS,
15  * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
16  * See the License for the specific language governing permissions and
17  * limitations under the License.
18  * ---license-end
19  */
20 
21 package app.coronawarn.server.services.distribution.config;
22 
23 import app.coronawarn.server.common.protocols.external.exposurenotification.SignatureInfo;
24 import javax.validation.constraints.Max;
25 import javax.validation.constraints.Min;
26 import javax.validation.constraints.Pattern;
27 import org.springframework.boot.context.properties.ConfigurationProperties;
28 import org.springframework.stereotype.Component;
29 import org.springframework.validation.annotation.Validated;
30 
31 @Component
32 @ConfigurationProperties(prefix = "services.distribution")
33 @Validated
34 public class DistributionServiceConfig {
35 
36   private static final String PATH_REGEX = "^[/]?[a-zA-Z0-9_]+(/[a-zA-Z0-9_]+)*[/]?$";
37   private static final String FILE_NAME_REGEX = "^[a-zA-Z0-9_-]+$";
38   private static final String FILE_NAME_WITH_TYPE_REGEX = "^[a-zA-Z0-9_-]+\\.[a-z]+$";
39   private static final String CHAR_AND_NUMBER_REGEX = "^[a-zA-Z0-9_-]+$";
40   private static final String CHAR_NUMBER_AND_SPACE_REGEX = "^[a-zA-Z0-9_\\s]+$";
41   private static final String NO_WHITESPACE_REGEX = "^[\\S]+$";
42   private static final String URL_REGEX = "^http[s]?://[a-z0-9-]+([\\./][a-z0-9-]+)*[/]?$";
43   private static final String NUMBER_REGEX = "^[0-9]+$";
44   private static final String VERSION_REGEX = "^v[0-9]+$";
45   private static final String ALGORITHM_OID_REGEX = "^[0-9]+[\\.[0-9]+]*$";
46   private static final String BUNDLE_REGEX = "^[a-z-]+[\\.[a-z-]+]*$";
47   private static final String PRIVATE_KEY_REGEX = "^(classpath:|file://)[/]?[a-zA-Z0-9_]+[/[a-zA-Z0-9_]+]*(.pem)?$";
48 
49   private Paths paths;
50   private TestData testData;
51   @Min(0)
52   @Max(28)
53   private Integer retentionDays;
54   @Min(120)
55   @Max(720)
56   private Integer expiryPolicyMinutes;
57   @Min(0)
58   @Max(200)
59   private Integer shiftingPolicyThreshold;
60   @Min(600000)
61   @Max(750000)
62   private Integer maximumNumberOfKeysPerBundle;
63   @Pattern(regexp = FILE_NAME_REGEX)
64   private String outputFileName;
65   private Boolean includeIncompleteDays;
66   private Boolean includeIncompleteHours;
67   private TekExport tekExport;
68   private Signature signature;
69   private Api api;
70   private ObjectStore objectStore;
71 
72   public Paths getPaths() {
73     return paths;
74   }
75 
76   public void setPaths(Paths paths) {
77     this.paths = paths;
78   }
79 
80   public TestData getTestData() {
81     return testData;
82   }
83 
84   public void setTestData(TestData testData) {
85     this.testData = testData;
86   }
87 
88   public Integer getRetentionDays() {
89     return retentionDays;
90   }
91 
92   public void setRetentionDays(Integer retentionDays) {
93     this.retentionDays = retentionDays;
94   }
95 
96   public Integer getExpiryPolicyMinutes() {
97     return expiryPolicyMinutes;
98   }
99 
100   public void setExpiryPolicyMinutes(Integer expiryPolicyMinutes) {
101     this.expiryPolicyMinutes = expiryPolicyMinutes;
102   }
103 
104   public Integer getShiftingPolicyThreshold() {
105     return shiftingPolicyThreshold;
106   }
107 
108   public void setShiftingPolicyThreshold(Integer shiftingPolicyThreshold) {
109     this.shiftingPolicyThreshold = shiftingPolicyThreshold;
110   }
111 
112   public Integer getMaximumNumberOfKeysPerBundle() {
113     return this.maximumNumberOfKeysPerBundle;
114   }
115 
116   public void setMaximumNumberOfKeysPerBundle(Integer maximumNumberOfKeysPerBundle) {
117     this.maximumNumberOfKeysPerBundle = maximumNumberOfKeysPerBundle;
118   }
119 
120   public String getOutputFileName() {
121     return outputFileName;
122   }
123 
124   public void setOutputFileName(String outputFileName) {
125     this.outputFileName = outputFileName;
126   }
127 
128   public Boolean getIncludeIncompleteDays() {
129     return includeIncompleteDays;
130   }
131 
132   public void setIncludeIncompleteDays(Boolean includeIncompleteDays) {
133     this.includeIncompleteDays = includeIncompleteDays;
134   }
135 
136   public Boolean getIncludeIncompleteHours() {
137     return includeIncompleteHours;
138   }
139 
140   public void setIncludeIncompleteHours(Boolean includeIncompleteHours) {
141     this.includeIncompleteHours = includeIncompleteHours;
142   }
143 
144   public TekExport getTekExport() {
145     return tekExport;
146   }
147 
148   public void setTekExport(TekExport tekExport) {
149     this.tekExport = tekExport;
150   }
151 
152   public Signature getSignature() {
153     return signature;
154   }
155 
156   public void setSignature(Signature signature) {
157     this.signature = signature;
158   }
159 
160   public Api getApi() {
161     return api;
162   }
163 
164   public void setApi(Api api) {
165     this.api = api;
166   }
167 
168   public ObjectStore getObjectStore() {
169     return objectStore;
170   }
171 
172   public void setObjectStore(
173       ObjectStore objectStore) {
174     this.objectStore = objectStore;
175   }
176 
177   public static class TekExport {
178 
179     @Pattern(regexp = FILE_NAME_WITH_TYPE_REGEX)
180     private String fileName;
181     @Pattern(regexp = CHAR_NUMBER_AND_SPACE_REGEX)
182     private String fileHeader;
183     @Min(0)
184     @Max(32)
185     private Integer fileHeaderWidth;
186 
187     public String getFileName() {
188       return fileName;
189     }
190 
191     public void setFileName(String fileName) {
192       this.fileName = fileName;
193     }
194 
195     public String getFileHeader() {
196       return fileHeader;
197     }
198 
199     public void setFileHeader(String fileHeader) {
200       this.fileHeader = fileHeader;
201     }
202 
203     public Integer getFileHeaderWidth() {
204       return fileHeaderWidth;
205     }
206 
207     public void setFileHeaderWidth(Integer fileHeaderWidth) {
208       this.fileHeaderWidth = fileHeaderWidth;
209     }
210   }
211 
212   public static class TestData {
213 
214     private Integer seed;
215     private Integer exposuresPerHour;
216 
217     public Integer getSeed() {
218       return seed;
219     }
220 
221     public void setSeed(Integer seed) {
222       this.seed = seed;
223     }
224 
225     public Integer getExposuresPerHour() {
226       return exposuresPerHour;
227     }
228 
229     public void setExposuresPerHour(Integer exposuresPerHour) {
230       this.exposuresPerHour = exposuresPerHour;
231     }
232   }
233 
234   public static class Paths {
235 
236     @Pattern(regexp = PRIVATE_KEY_REGEX)
237     private String privateKey;
238     @Pattern(regexp = PATH_REGEX)
239     private String output;
240 
241     public String getPrivateKey() {
242       return privateKey;
243     }
244 
245     public void setPrivateKey(String privateKey) {
246       this.privateKey = privateKey;
247     }
248 
249     public String getOutput() {
250       return output;
251     }
252 
253     public void setOutput(String output) {
254       this.output = output;
255     }
256   }
257 
258   public static class Api {
259 
260     @Pattern(regexp = CHAR_AND_NUMBER_REGEX)
261     private String versionPath;
262     @Pattern(regexp = VERSION_REGEX)
263     private String versionV1;
264     @Pattern(regexp = CHAR_AND_NUMBER_REGEX)
265     private String countryPath;
266     @Pattern(regexp = CHAR_AND_NUMBER_REGEX)
267     private String countryGermany;
268     @Pattern(regexp = CHAR_AND_NUMBER_REGEX)
269     private String datePath;
270     @Pattern(regexp = CHAR_AND_NUMBER_REGEX)
271     private String hourPath;
272     @Pattern(regexp = CHAR_AND_NUMBER_REGEX)
273     private String diagnosisKeysPath;
274     @Pattern(regexp = CHAR_AND_NUMBER_REGEX)
275     private String parametersPath;
276     @Pattern(regexp = CHAR_AND_NUMBER_REGEX)
277     private String appConfigFileName;
278 
279     public String getVersionPath() {
280       return versionPath;
281     }
282 
283     public void setVersionPath(String versionPath) {
284       this.versionPath = versionPath;
285     }
286 
287     public String getVersionV1() {
288       return versionV1;
289     }
290 
291     public void setVersionV1(String versionV1) {
292       this.versionV1 = versionV1;
293     }
294 
295     public String getCountryPath() {
296       return countryPath;
297     }
298 
299     public void setCountryPath(String countryPath) {
300       this.countryPath = countryPath;
301     }
302 
303     public String getCountryGermany() {
304       return countryGermany;
305     }
306 
307     public void setCountryGermany(String countryGermany) {
308       this.countryGermany = countryGermany;
309     }
310 
311     public String getDatePath() {
312       return datePath;
313     }
314 
315     public void setDatePath(String datePath) {
316       this.datePath = datePath;
317     }
318 
319     public String getHourPath() {
320       return hourPath;
321     }
322 
323     public void setHourPath(String hourPath) {
324       this.hourPath = hourPath;
325     }
326 
327     public String getDiagnosisKeysPath() {
328       return diagnosisKeysPath;
329     }
330 
331     public void setDiagnosisKeysPath(String diagnosisKeysPath) {
332       this.diagnosisKeysPath = diagnosisKeysPath;
333     }
334 
335     public String getParametersPath() {
336       return parametersPath;
337     }
338 
339     public void setParametersPath(String parametersPath) {
340       this.parametersPath = parametersPath;
341     }
342 
343     public String getAppConfigFileName() {
344       return appConfigFileName;
345     }
346 
347     public void setAppConfigFileName(String appConfigFileName) {
348       this.appConfigFileName = appConfigFileName;
349     }
350   }
351 
352   public static class Signature {
353 
354     @Pattern(regexp = BUNDLE_REGEX)
355     private String appBundleId;
356     private String androidPackage;
357     @Pattern(regexp = NUMBER_REGEX)
358     private String verificationKeyId;
359     @Pattern(regexp = VERSION_REGEX)
360     private String verificationKeyVersion;
361     @Pattern(regexp = ALGORITHM_OID_REGEX)
362     private String algorithmOid;
363     @Pattern(regexp = CHAR_AND_NUMBER_REGEX)
364     private String algorithmName;
365     @Pattern(regexp = FILE_NAME_WITH_TYPE_REGEX)
366     private String fileName;
367     @Pattern(regexp = CHAR_AND_NUMBER_REGEX)
368     private String securityProvider;
369 
370     public String getAppBundleId() {
371       return appBundleId;
372     }
373 
374     public void setAppBundleId(String appBundleId) {
375       this.appBundleId = appBundleId;
376     }
377 
378     public String getAndroidPackage() {
379       return androidPackage;
380     }
381 
382     public void setAndroidPackage(String androidPackage) {
383       this.androidPackage = androidPackage;
384     }
385 
386     public String getVerificationKeyId() {
387       return verificationKeyId;
388     }
389 
390     public void setVerificationKeyId(String verificationKeyId) {
391       this.verificationKeyId = verificationKeyId;
392     }
393 
394     public String getVerificationKeyVersion() {
395       return verificationKeyVersion;
396     }
397 
398     public void setVerificationKeyVersion(String verificationKeyVersion) {
399       this.verificationKeyVersion = verificationKeyVersion;
400     }
401 
402     public String getAlgorithmOid() {
403       return algorithmOid;
404     }
405 
406     public void setAlgorithmOid(String algorithmOid) {
407       this.algorithmOid = algorithmOid;
408     }
409 
410     public String getAlgorithmName() {
411       return algorithmName;
412     }
413 
414     public void setAlgorithmName(String algorithmName) {
415       this.algorithmName = algorithmName;
416     }
417 
418     public String getFileName() {
419       return fileName;
420     }
421 
422     public void setFileName(String fileName) {
423       this.fileName = fileName;
424     }
425 
426     public String getSecurityProvider() {
427       return securityProvider;
428     }
429 
430     public void setSecurityProvider(String securityProvider) {
431       this.securityProvider = securityProvider;
432     }
433 
434     /**
435      * Returns the static {@link SignatureInfo} configured in the application properties.
436      */
437     public SignatureInfo getSignatureInfo() {
438       return SignatureInfo.newBuilder()
439           .setAppBundleId(this.getAppBundleId())
440           .setVerificationKeyVersion(this.getVerificationKeyVersion())
441           .setVerificationKeyId(this.getVerificationKeyId())
442           .setSignatureAlgorithm(this.getAlgorithmOid())
443           .build();
444     }
445   }
446 
447   public static class ObjectStore {
448 
449     @Pattern(regexp = NO_WHITESPACE_REGEX)
450     private String accessKey;
451     @Pattern(regexp = NO_WHITESPACE_REGEX)
452     private String secretKey;
453     @Pattern(regexp = URL_REGEX)
454     private String endpoint;
455     @Min(1)
456     @Max(65535)
457     private Integer port;
458     @Pattern(regexp = CHAR_AND_NUMBER_REGEX)
459     private String bucket;
460     private Boolean setPublicReadAclOnPutObject;
461     @Min(1)
462     @Max(64)
463     private Integer maxNumberOfFailedOperations;
464     @Min(1)
465     @Max(64)
466     private Integer maxNumberOfS3Threads;
467 
468     public String getAccessKey() {
469       return accessKey;
470     }
471 
472     public void setAccessKey(String accessKey) {
473       this.accessKey = accessKey;
474     }
475 
476     public String getSecretKey() {
477       return secretKey;
478     }
479 
480     public void setSecretKey(String secretKey) {
481       this.secretKey = secretKey;
482     }
483 
484     public String getEndpoint() {
485       return endpoint;
486     }
487 
488     public void setEndpoint(String endpoint) {
489       this.endpoint = endpoint;
490     }
491 
492     public Integer getPort() {
493       return port;
494     }
495 
496     public void setPort(Integer port) {
497       this.port = port;
498     }
499 
500     public String getBucket() {
501       return bucket;
502     }
503 
504     public void setBucket(String bucket) {
505       this.bucket = bucket;
506     }
507 
508     public Boolean isSetPublicReadAclOnPutObject() {
509       return setPublicReadAclOnPutObject;
510     }
511 
512     public void setSetPublicReadAclOnPutObject(Boolean setPublicReadAclOnPutObject) {
513       this.setPublicReadAclOnPutObject = setPublicReadAclOnPutObject;
514     }
515 
516     public Integer getMaxNumberOfFailedOperations() {
517       return maxNumberOfFailedOperations;
518     }
519 
520     public void setMaxNumberOfFailedOperations(Integer maxNumberOfFailedOperations) {
521       this.maxNumberOfFailedOperations = maxNumberOfFailedOperations;
522     }
523 
524     public Integer getMaxNumberOfS3Threads() {
525       return maxNumberOfS3Threads;
526     }
527 
528     public void setMaxNumberOfS3Threads(Integer maxNumberOfS3Threads) {
529       this.maxNumberOfS3Threads = maxNumberOfS3Threads;
530     }
531   }
532 }
533 
[end of services/distribution/src/main/java/app/coronawarn/server/services/distribution/config/DistributionServiceConfig.java]
[start of services/distribution/src/main/java/app/coronawarn/server/services/distribution/objectstore/S3Publisher.java]
1 /*-
2  * ---license-start
3  * Corona-Warn-App
4  * ---
5  * Copyright (C) 2020 SAP SE and all other contributors
6  * ---
7  * Licensed under the Apache License, Version 2.0 (the "License");
8  * you may not use this file except in compliance with the License.
9  * You may obtain a copy of the License at
10  *
11  *      http://www.apache.org/licenses/LICENSE-2.0
12  *
13  * Unless required by applicable law or agreed to in writing, software
14  * distributed under the License is distributed on an "AS IS" BASIS,
15  * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
16  * See the License for the specific language governing permissions and
17  * limitations under the License.
18  * ---license-end
19  */
20 
21 package app.coronawarn.server.services.distribution.objectstore;
22 
23 import app.coronawarn.server.services.distribution.config.DistributionServiceConfig;
24 import app.coronawarn.server.services.distribution.objectstore.client.ObjectStoreOperationFailedException;
25 import app.coronawarn.server.services.distribution.objectstore.publish.LocalFile;
26 import app.coronawarn.server.services.distribution.objectstore.publish.PublishFileSet;
27 import app.coronawarn.server.services.distribution.objectstore.publish.PublishedFileSet;
28 import java.io.IOException;
29 import java.nio.file.Path;
30 import java.util.List;
31 import java.util.concurrent.ExecutionException;
32 import java.util.concurrent.Future;
33 import java.util.stream.Collectors;
34 import org.slf4j.Logger;
35 import org.slf4j.LoggerFactory;
36 import org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor;
37 import org.springframework.stereotype.Component;
38 
39 /**
40  * Publishes a folder on the disk to S3 while keeping the folder and file structure.<br> Moreover, does the following:
41  * <br>
42  * <ul>
43  *   <li>Publishes index files on a different route, removing the trailing "/index" part.</li>
44  *   <li>Adds meta information to the uploaded files, e.g. the sha256 hash value.</li>
45  *   <li>Only performs the upload for files, which do not yet exist on the object store, and
46  *   checks whether the existing files hash differ from the to-be-uploaded files hash. Only if the
47  *   hash differs, the file will ultimately be uploaded</li>
48  *   <li>Currently not implemented: Set cache control headers</li>
49  *   <li>Currently not implemented: Supports multi threaded upload of files.</li>
50  * </ul>
51  */
52 @Component
53 public class S3Publisher {
54 
55   private static final Logger logger = LoggerFactory.getLogger(S3Publisher.class);
56 
57   private final ObjectStoreAccess objectStoreAccess;
58   private final FailedObjectStoreOperationsCounter failedOperationsCounter;
59   private final ThreadPoolTaskExecutor executor;
60   private final DistributionServiceConfig distributionServiceConfig;
61 
62   /**
63    * Creates an {@link S3Publisher} instance that attempts to publish the files at the specified location to an object
64    * store. Object store operations are performed through the specified {@link ObjectStoreAccess} instance.
65    *
66    * @param objectStoreAccess         The {@link ObjectStoreAccess} used to communicate with the object store.
67    * @param failedOperationsCounter   The {@link FailedObjectStoreOperationsCounter} that is used to monitor the number
68    *                                  of failed operations.
69    * @param executor                  The executor that manages the upload task submission.
70    * @param distributionServiceConfig The {@link DistributionServiceConfig} used for distribution service
71    *                                  configuration.
72    */
73   public S3Publisher(ObjectStoreAccess objectStoreAccess, FailedObjectStoreOperationsCounter failedOperationsCounter,
74       ThreadPoolTaskExecutor executor, DistributionServiceConfig distributionServiceConfig) {
75     this.objectStoreAccess = objectStoreAccess;
76     this.failedOperationsCounter = failedOperationsCounter;
77     this.executor = executor;
78     this.distributionServiceConfig = distributionServiceConfig;
79   }
80 
81   /**
82    * Synchronizes the files to S3.
83    *
84    * @param root The path of the directory that shall be published.
85    * @throws IOException in case there were problems reading files from the disk.
86    */
87   public void publish(Path root) throws IOException {
88     PublishedFileSet published;
89     List<LocalFile> toPublish = new PublishFileSet(root).getFiles();
90     List<LocalFile> diff;
91 
92     try {
93       published = new PublishedFileSet(
94           objectStoreAccess.getObjectsWithPrefix(distributionServiceConfig.getApi().getVersionPath()));
95       diff = toPublish
96           .stream()
97           .filter(published::isNotYetPublished)
98           .collect(Collectors.toList());
99     } catch (ObjectStoreOperationFailedException e) {
100       failedOperationsCounter.incrementAndCheckThreshold(e);
101       // failed to retrieve existing files; publish everything
102       diff = toPublish;
103     }
104 
105     logger.info("Beginning upload of {} files... ", diff.size());
106     try {
107       diff.stream()
108           .map(file -> executor.submit(() -> objectStoreAccess.putObject(file)))
109           .forEach(this::awaitThread);
110     } finally {
111       executor.shutdown();
112     }
113     logger.info("Upload completed.");
114   }
115 
116   private void awaitThread(Future<?> result) {
117     try {
118       result.get();
119     } catch (ExecutionException e) {
120       failedOperationsCounter.incrementAndCheckThreshold(new ObjectStoreOperationFailedException(e.getMessage(), e));
121     } catch (InterruptedException e) {
122       Thread.currentThread().interrupt();
123       throw new ObjectStoreOperationFailedException(e.getMessage(), e);
124     }
125   }
126 }
127 
[end of services/distribution/src/main/java/app/coronawarn/server/services/distribution/objectstore/S3Publisher.java]
[start of services/distribution/src/main/java/app/coronawarn/server/services/distribution/objectstore/client/S3Object.java]
1 /*-
2  * ---license-start
3  * Corona-Warn-App
4  * ---
5  * Copyright (C) 2020 SAP SE and all other contributors
6  * ---
7  * Licensed under the Apache License, Version 2.0 (the "License");
8  * you may not use this file except in compliance with the License.
9  * You may obtain a copy of the License at
10  *
11  *      http://www.apache.org/licenses/LICENSE-2.0
12  *
13  * Unless required by applicable law or agreed to in writing, software
14  * distributed under the License is distributed on an "AS IS" BASIS,
15  * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
16  * See the License for the specific language governing permissions and
17  * limitations under the License.
18  * ---license-end
19  */
20 
21 package app.coronawarn.server.services.distribution.objectstore.client;
22 
23 import java.util.Objects;
24 
25 /**
26  * Represents an object as discovered on S3.
27  */
28 public class S3Object {
29 
30   /**
31    * the name of the object.
32    */
33   private final String objectName;
34 
35   /** The cwaHash of this S3 Object. */
36   private String cwaHash;
37 
38   /**
39    * Constructs a new S3Object for the given object name.
40    *
41    * @param objectName the target object name
42    */
43   public S3Object(String objectName) {
44     this.objectName = objectName;
45   }
46 
47   /**
48    * Constructs a new S3Object for the given object name.
49    *
50    * @param objectName the target object name
51    * @param cwaHash the checksum for that file
52    */
53   public S3Object(String objectName, String cwaHash) {
54     this(objectName);
55     this.cwaHash = cwaHash;
56   }
57 
58   public String getObjectName() {
59     return objectName;
60   }
61 
62   public String getCwaHash() {
63     return cwaHash;
64   }
65 
66   @Override
67   public boolean equals(Object o) {
68     if (this == o) {
69       return true;
70     }
71     if (o == null || getClass() != o.getClass()) {
72       return false;
73     }
74     S3Object s3Object = (S3Object) o;
75     return Objects.equals(objectName, s3Object.objectName) && Objects.equals(cwaHash, s3Object.cwaHash);
76   }
77 
78   @Override
79   public int hashCode() {
80     return Objects.hash(objectName, cwaHash);
81   }
82 }
83 
[end of services/distribution/src/main/java/app/coronawarn/server/services/distribution/objectstore/client/S3Object.java]
[start of services/distribution/src/main/java/app/coronawarn/server/services/distribution/objectstore/publish/LocalFile.java]
1 /*-
2  * ---license-start
3  * Corona-Warn-App
4  * ---
5  * Copyright (C) 2020 SAP SE and all other contributors
6  * ---
7  * Licensed under the Apache License, Version 2.0 (the "License");
8  * you may not use this file except in compliance with the License.
9  * You may obtain a copy of the License at
10  *
11  *      http://www.apache.org/licenses/LICENSE-2.0
12  *
13  * Unless required by applicable law or agreed to in writing, software
14  * distributed under the License is distributed on an "AS IS" BASIS,
15  * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
16  * See the License for the specific language governing permissions and
17  * limitations under the License.
18  * ---license-end
19  */
20 
21 package app.coronawarn.server.services.distribution.objectstore.publish;
22 
23 import app.coronawarn.server.services.distribution.assembly.structure.file.FileOnDiskWithChecksum;
24 import java.io.IOException;
25 import java.nio.file.Files;
26 import java.nio.file.Path;
27 import org.slf4j.Logger;
28 import org.slf4j.LoggerFactory;
29 
30 /**
31  * Represents a file, which is subject for publishing to S3.
32  */
33 public abstract class LocalFile {
34 
35   private static final Logger logger = LoggerFactory.getLogger(LocalFile.class);
36 
37   /**
38    * the path to the file to be represented.
39    */
40   private final Path file;
41 
42   /**
43    * the assigned S3 key.
44    */
45   private final String s3Key;
46 
47   /**
48    * the checksum of this file.
49    */
50   private String checksum = "";
51 
52   /**
53    * Constructs a new file representing a file on the disk.
54    *
55    * @param file     the path to the file to be represented
56    * @param basePath the base path
57    */
58   public LocalFile(Path file, Path basePath) {
59     this.file = file;
60     this.s3Key = createS3Key(file, basePath);
61     this.checksum = loadChecksum();
62   }
63 
64   public String getS3Key() {
65     return s3Key;
66   }
67 
68   public String getChecksum() {
69     return checksum;
70   }
71 
72   public Path getFile() {
73     return file;
74   }
75 
76   private String loadChecksum() {
77     try {
78       return Files.readString(FileOnDiskWithChecksum.buildChecksumPathForFile(file)).trim();
79     } catch (IOException e) {
80       logger.debug("Unable to load checksum file.");
81       return "";
82     }
83   }
84 
85   protected String createS3Key(Path file, Path rootFolder) {
86     Path relativePath = rootFolder.relativize(file);
87     return relativePath.toString().replaceAll("\\\\", "/");
88   }
89 
90   /**
91    * Value for the <code>content-type</code> header.
92    * 
93    * @return Either <a href="https://www.iana.org/assignments/media-types/application/zip">zip</a> or
94    *         <a href="https://www.iana.org/assignments/media-types/application/json">json</a>.
95    */
96   public String getContentType() {
97     if (s3Key.endsWith("app_config")) {
98       return "application/zip";
99     }
100     if (s3Key.matches(".*\\d")) {
101       // date and hourly diagnosis key files
102       return "application/zip";
103     }
104     // list of versions, dates, hours
105     return "application/json";
106   }
107 }
108 
[end of services/distribution/src/main/java/app/coronawarn/server/services/distribution/objectstore/publish/LocalFile.java]
[start of services/distribution/src/main/java/app/coronawarn/server/services/distribution/objectstore/publish/PublishedFileSet.java]
1 /*-
2  * ---license-start
3  * Corona-Warn-App
4  * ---
5  * Copyright (C) 2020 SAP SE and all other contributors
6  * ---
7  * Licensed under the Apache License, Version 2.0 (the "License");
8  * you may not use this file except in compliance with the License.
9  * You may obtain a copy of the License at
10  *
11  *      http://www.apache.org/licenses/LICENSE-2.0
12  *
13  * Unless required by applicable law or agreed to in writing, software
14  * distributed under the License is distributed on an "AS IS" BASIS,
15  * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
16  * See the License for the specific language governing permissions and
17  * limitations under the License.
18  * ---license-end
19  */
20 
21 package app.coronawarn.server.services.distribution.objectstore.publish;
22 
23 import app.coronawarn.server.services.distribution.objectstore.client.S3Object;
24 import java.util.List;
25 import java.util.Map;
26 import java.util.stream.Collectors;
27 
28 /**
29  * Provides an overview about which files are currently available on S3.
30  */
31 public class PublishedFileSet {
32 
33   /** ta map of S3 objects with the S3 object name as the key component of the map. */
34   private Map<String, S3Object> s3Objects;
35 
36   /**
37    * Creates a new PublishedFileSet for the given S3 objects with the help of the metadata provider.
38    * The metadata provider helps to determine whether files have been changed, and are requiring
39    * re-upload.
40    *
41    * @param s3Objects the list of s3 objects.
42    */
43   public PublishedFileSet(List<S3Object> s3Objects) {
44     this.s3Objects = s3Objects.stream()
45         .collect(Collectors.toMap(S3Object::getObjectName, s3object -> s3object));
46   }
47 
48   /**
49    * Checks whether the given file, which is subject for publishing, is already available on the S3.
50    * Will return true, when:
51    * <ul>
52    *   <li>The S3 object key exists on S3</li>
53    *   <li>The checksum of the existing S3 object matches the hash of the given file</li>
54    * </ul>
55    *
56    * @param file the to-be-published file which should be checked
57    * @return true, if it exists and is identical
58    */
59   public boolean isNotYetPublished(LocalFile file) {
60     S3Object published = s3Objects.get(file.getS3Key());
61 
62     if (published == null) {
63       return true;
64     }
65 
66     return !file.getChecksum().equals(published.getCwaHash());
67   }
68 
69 }
70 
[end of services/distribution/src/main/java/app/coronawarn/server/services/distribution/objectstore/publish/PublishedFileSet.java]
[start of services/distribution/src/main/java/app/coronawarn/server/services/distribution/runner/RetentionPolicy.java]
1 /*-
2  * ---license-start
3  * Corona-Warn-App
4  * ---
5  * Copyright (C) 2020 SAP SE and all other contributors
6  * ---
7  * Licensed under the Apache License, Version 2.0 (the "License");
8  * you may not use this file except in compliance with the License.
9  * You may obtain a copy of the License at
10  *
11  *      http://www.apache.org/licenses/LICENSE-2.0
12  *
13  * Unless required by applicable law or agreed to in writing, software
14  * distributed under the License is distributed on an "AS IS" BASIS,
15  * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
16  * See the License for the specific language governing permissions and
17  * limitations under the License.
18  * ---license-end
19  */
20 
21 package app.coronawarn.server.services.distribution.runner;
22 
23 import app.coronawarn.server.common.persistence.service.DiagnosisKeyService;
24 import app.coronawarn.server.services.distribution.Application;
25 import app.coronawarn.server.services.distribution.config.DistributionServiceConfig;
26 import app.coronawarn.server.services.distribution.objectstore.S3RetentionPolicy;
27 import org.slf4j.Logger;
28 import org.slf4j.LoggerFactory;
29 import org.springframework.boot.ApplicationArguments;
30 import org.springframework.boot.ApplicationRunner;
31 import org.springframework.context.ApplicationContext;
32 import org.springframework.core.annotation.Order;
33 import org.springframework.stereotype.Component;
34 
35 /**
36  * This runner removes any diagnosis keys from the database that were submitted before a configured threshold of days.
37  */
38 @Component
39 @Order(1)
40 public class RetentionPolicy implements ApplicationRunner {
41 
42   private static final Logger logger = LoggerFactory
43       .getLogger(RetentionPolicy.class);
44 
45   private final DiagnosisKeyService diagnosisKeyService;
46 
47   private final ApplicationContext applicationContext;
48 
49   private final Integer retentionDays;
50 
51   private final S3RetentionPolicy s3RetentionPolicy;
52 
53   /**
54    * Creates a new RetentionPolicy.
55    */
56   RetentionPolicy(DiagnosisKeyService diagnosisKeyService,
57       ApplicationContext applicationContext,
58       DistributionServiceConfig distributionServiceConfig,
59       S3RetentionPolicy s3RetentionPolicy) {
60     this.diagnosisKeyService = diagnosisKeyService;
61     this.applicationContext = applicationContext;
62     this.retentionDays = distributionServiceConfig.getRetentionDays();
63     this.s3RetentionPolicy = s3RetentionPolicy;
64   }
65 
66   @Override
67   public void run(ApplicationArguments args) {
68     try {
69       diagnosisKeyService.applyRetentionPolicy(retentionDays);
70       s3RetentionPolicy.applyRetentionPolicy(retentionDays);
71     } catch (Exception e) {
72       logger.error("Application of retention policy failed.", e);
73       Application.killApplication(applicationContext);
74     }
75 
76     logger.debug("Retention policy applied successfully.");
77   }
78 }
79 
[end of services/distribution/src/main/java/app/coronawarn/server/services/distribution/runner/RetentionPolicy.java]
[start of services/distribution/src/main/java/app/coronawarn/server/services/distribution/runner/S3Distribution.java]
1 /*-
2  * ---license-start
3  * Corona-Warn-App
4  * ---
5  * Copyright (C) 2020 SAP SE and all other contributors
6  * ---
7  * Licensed under the Apache License, Version 2.0 (the "License");
8  * you may not use this file except in compliance with the License.
9  * You may obtain a copy of the License at
10  *
11  *      http://www.apache.org/licenses/LICENSE-2.0
12  *
13  * Unless required by applicable law or agreed to in writing, software
14  * distributed under the License is distributed on an "AS IS" BASIS,
15  * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
16  * See the License for the specific language governing permissions and
17  * limitations under the License.
18  * ---license-end
19  */
20 
21 package app.coronawarn.server.services.distribution.runner;
22 
23 import app.coronawarn.server.services.distribution.assembly.component.OutputDirectoryProvider;
24 import app.coronawarn.server.services.distribution.objectstore.S3Publisher;
25 import app.coronawarn.server.services.distribution.objectstore.client.ObjectStoreOperationFailedException;
26 import java.io.IOException;
27 import java.nio.file.Path;
28 import org.slf4j.Logger;
29 import org.slf4j.LoggerFactory;
30 import org.springframework.boot.ApplicationArguments;
31 import org.springframework.boot.ApplicationRunner;
32 import org.springframework.core.annotation.Order;
33 import org.springframework.stereotype.Component;
34 
35 /**
36  * This runner will sync the base working directory to the S3.
37  */
38 @Component
39 @Order(3)
40 public class S3Distribution implements ApplicationRunner {
41 
42   private static final Logger logger = LoggerFactory.getLogger(S3Distribution.class);
43 
44   private final OutputDirectoryProvider outputDirectoryProvider;
45   private final S3Publisher s3Publisher;
46 
47   S3Distribution(OutputDirectoryProvider outputDirectoryProvider, S3Publisher s3Publisher) {
48     this.outputDirectoryProvider = outputDirectoryProvider;
49     this.s3Publisher = s3Publisher;
50   }
51 
52   @Override
53   public void run(ApplicationArguments args) {
54     try {
55       Path pathToDistribute = outputDirectoryProvider.getFileOnDisk().toPath().toAbsolutePath();
56 
57       s3Publisher.publish(pathToDistribute);
58       logger.info("Data pushed to Object Store successfully.");
59     } catch (UnsupportedOperationException | ObjectStoreOperationFailedException | IOException e) {
60       logger.error("Distribution failed.", e);
61     }
62   }
63 }
64 
[end of services/distribution/src/main/java/app/coronawarn/server/services/distribution/runner/S3Distribution.java]
[start of services/distribution/src/main/resources/application.yaml]
1 ---
2 logging:
3   level:
4     org:
5       springframework:
6         web: INFO
7     app:
8       coronawarn: INFO
9 
10 services:
11   distribution:
12     # The name of the distribution output file.
13     output-file-name: index
14     # The number of days to retain diagnosis keys for both database persistency layer and files stored on the object store.
15     retention-days: 14
16     # The number of minutes that diagnosis keys must have been expired for (since the end of the rolling interval window) before they can be distributed.
17     expiry-policy-minutes: 120
18     # The minimum number of diagnosis keys per bundle.
19     shifting-policy-threshold: 140
20     # The maximum number of keys per bundle.
21     maximum-number-of-keys-per-bundle: 600000
22     # Indicates whether the current incomplete day will be included in the distribution (used for testing purposes).
23     include-incomplete-days: false
24     # Indicates whether the current incomplete hour will be included in the distribution (used for testing purposes).
25     include-incomplete-hours: false
26     # Local paths, that are used during the export creation.
27     paths:
28       # The output path.
29       output: out
30       # The location of the private key.
31       privatekey: ${VAULT_FILESIGNING_SECRET}
32     # Configuration for the exported archive, that is saved on the S3-compatible storage.
33     tek-export:
34       # The TEK file name included in the zip archive, containing the list of diagnosis keys.
35       file-name: export.bin
36       # The TEK file header.
37       file-header: EK Export v1
38       # The fixed (ensured by right whitespace padding) TEK file header width.
39       file-header-width: 16
40     # Configuration for the API which is used by the mobile app to query diagnosis keys.
41     api:
42       version-path: version
43       version-v1: v1
44       country-path: country
45       country-germany: DE
46       date-path: date
47       hour-path: hour
48       diagnosis-keys-path: diagnosis-keys
49       parameters-path: configuration
50       app-config-file-name: app_config
51     # Signature configuration, used for signing the exports.
52     signature:
53       # The alias with which to identify public key to be used for verification.
54       verification-key-id: 262
55       # The key version for rollovers.
56       verification-key-version: v1
57       # The ASN.1 OID for algorithm identifier.
58       algorithm-oid: 1.2.840.10045.4.3.2
59       # The algorithm name.
60       algorithm-name: SHA256withECDSA
61       # The signature file name included in the zip archive.
62       file-name: export.sig
63       # The security provider.
64       security-provider: BC
65     # Configuration for the S3 compatible object storage hosted by Telekom in Germany.
66     objectstore:
67       access-key: ${CWA_OBJECTSTORE_ACCESSKEY:accessKey1}
68       secret-key: ${CWA_OBJECTSTORE_SECRETKEY:verySecretKey1}
69       endpoint: ${CWA_OBJECTSTORE_ENDPOINT:http://localhost}
70       bucket: ${CWA_OBJECTSTORE_BUCKET:cwa}
71       port: ${CWA_OBJECTSTORE_PORT:8003}
72       # Indicates whether the S3 Access Control List (ACL) header should be set to 'public-read' on put object.
73       set-public-read-acl-on-put-object: true
74       # The number of maximum retry attempts used for configuring Spring @Retryable annotation.
75       retry-attempts: 3
76       # The backoff period in milliseconds used for configuring Spring @Retryable annotation.
77       retry-backoff: 2000
78       # The maximum number of failed operations before giving up.
79       max-number-of-failed-operations: 5
80       # The ThreadPoolTaskExecutor's maximum thread pool size.
81       max-number-of-s3-threads: 4
82 
83 spring:
84   main:
85     web-application-type: NONE
86   # Postgres configuration
87   flyway:
88     enabled: true
89     locations: classpath:/db/migration, classpath:/db/specific/{vendor}
90     password: ${POSTGRESQL_PASSWORD_FLYWAY:local_setup_flyway}
91     user: ${POSTGRESQL_USER_FLYWAY:local_setup_flyway}
92 
93   datasource:
94     driver-class-name: org.postgresql.Driver
95     url: jdbc:postgresql://${POSTGRESQL_SERVICE_HOST}:${POSTGRESQL_SERVICE_PORT}/${POSTGRESQL_DATABASE}?ssl=true&sslmode=verify-full&sslrootcert=${SSL_POSTGRES_CERTIFICATE_PATH}&sslcert=${SSL_DISTRIBUTION_CERTIFICATE_PATH}&sslkey=${SSL_DISTRIBUTION_PRIVATE_KEY_PATH}
96     username: ${POSTGRESQL_USER_DISTRIBUTION:local_setup_distribution}
97     password: ${POSTGRESQL_PASSWORD_DISTRIBUTION:local_setup_distribution}
98 
[end of services/distribution/src/main/resources/application.yaml]
</code>

Here is an example of a patch file. It consists of changes to the code base. It specifies the file names, the line numbers of each change, and the removed and added lines. A single patch file can contain changes to multiple files.
<patch>
--- a/file.py
+++ b/file.py
@@ -1,27 +1,35 @@
 def euclidean(a, b):
-    while b:
-        a, b = b, a % b
-    return a
+    if b == 0:
+        return a
+    return euclidean(b, a % b)
 
 
 def bresenham(x0, y0, x1, y1):
     points = []
     dx = abs(x1 - x0)
     dy = abs(y1 - y0)
-    sx = 1 if x0 < x1 else -1
-    sy = 1 if y0 < y1 else -1
-    err = dx - dy
+    x, y = x0, y0
+    sx = -1 if x0 > x1 else 1
+    sy = -1 if y0 > y1 else 1
 
-    while True:
-        points.append((x0, y0))
-        if x0 == x1 and y0 == y1:
-            break
-        e2 = 2 * err
-        if e2 > -dy:
+    if dx > dy:
+        err = dx / 2.0
+        while x != x1:
+            points.append((x, y))
             err -= dy
-            x0 += sx
-        if e2 < dx:
-            err += dx
-            y0 += sy
+            if err < 0:
+                y += sy
+                err += dx
+            x += sx
+    else:
+        err = dy / 2.0
+        while y != y1:
+            points.append((x, y))
+            err -= dx
+            if err < 0:
+                x += sx
+                err += dy
+            y += sy
 
+    points.append((x, y))
     return points
</patch>

I need you to solve the provded issue by generating a single patch file that I can apply directly to this repository using git apply. Please respond with a single patch file in the format shown above.
Respond below:


Wrong hash value for 2020-06-23 and 2020-06-24
<!--
Thanks for proposing an enhancement 🙌 ❤️

Before opening a new issue, please make sure that we do not have any duplicates already open. You can ensure this by searching the issue list for this repository. If there is a duplicate, please close your issue and add a comment to the existing issue instead.
-->


## Current Implementation
<!-- Describe or point to the current implementation that you would like to see improved -->
Old keys are deleted and the remaining keys are repacked by day or by hour.
60 keys moved from 24.06.2020 to 23.06.2020.
44 keys are deleted from 23.06.2020
The hour files 2020-06-23-hour-08.zip, 2020-06-23-hour-13.zip, 2020-06-23-hour-17.zip are repacked to
2020-06-23-hour-10.zip, 2020-06-23-hour-15.zip, 2020-06-23-hour-18.zip
(Even the old hour files are superseded they are still on the server.)
## Suggested Enhancement
<!-- Outline the idea of your enhancement, by e.g., describing the algorithm you propose. You can also create a Pull Request to outline your idea -->
Leave the files as they are, do not remove old keys. 
## Expected Benefits
<!-- Summarize how your enhancement could aid the implementation (performance, readability, memory consumption, battery consumption, etc.). Please also back up with measurements or give detailed explanations for reduced runtimes, memory consumption, etc.  -->
The files downloaded by the app have the same hash value as the file on the server.

Initial suspicion: Retention Policy kicked in and deleted older keys, which were not immediately distributed due to shifting policy. This in turn did lead to even less keys for the first days of CWA so the other packages were now also shifted.

<del>I believe this only affects hourly packages as retention enforcement & daily package creation are part of the same job. </del>
<del>
There are some options on how this can be fixed:</del>
1. <del>(Temporary) Disable hourly files. As they are currently not being used by the mobile app, there would be no impact.</del>
2. <del>Run retention only once per day and not on every job. Can be problematic in case this one job did not trigger or failed, so the next retention run would only occur after 24 hours. Not a big issue though.</del>
3. Store some pointer/information on the database which indicates which keys were already distributed. There are some edge cases which make this approach more complicated though (e.g. shifting keys due to 2 hours key validity expiration).

Edit: Daily files are just as affected as hourly files. Guess this leaves only one option.  @pithumke what do you think?

Today 2020-06-24.zip on the server has the same values and hash as it was first publish (so the 60 moved keys moved back).

2020-06-23.zip does not contain 173 keys of the original file. But this time, on IOS, the file has been downloaded again, why??
The hash and the # of keys are the same in IOS and on the server (330 keys now). 

All keys publish in 2020-06-23-hour-08.zip are missing in the daily file 2020-06-23.zip, now. 
Is this expected?


<patch>
diff --git a/docker-compose.yml b/docker-compose.yml
--- a/docker-compose.yml
+++ b/docker-compose.yml
@@ -46,6 +46,7 @@ services:
       services.distribution.paths.output: /tmp/distribution
       # Settings for cryptographic artifacts
       VAULT_FILESIGNING_SECRET: ${SECRET_PRIVATE}
+      FORCE_UPDATE_KEYFILES: 'false'
     volumes:
       - ./docker-compose-test-secrets:/secrets
   postgres:
diff --git a/services/distribution/src/main/java/app/coronawarn/server/services/distribution/assembly/diagnosiskeys/ProdDiagnosisKeyBundler.java b/services/distribution/src/main/java/app/coronawarn/server/services/distribution/assembly/diagnosiskeys/ProdDiagnosisKeyBundler.java
--- a/services/distribution/src/main/java/app/coronawarn/server/services/distribution/assembly/diagnosiskeys/ProdDiagnosisKeyBundler.java
+++ b/services/distribution/src/main/java/app/coronawarn/server/services/distribution/assembly/diagnosiskeys/ProdDiagnosisKeyBundler.java
@@ -31,6 +31,7 @@
 import java.time.temporal.ChronoUnit;
 import java.util.ArrayList;
 import java.util.Collection;
+import java.util.Collections;
 import java.util.HashMap;
 import java.util.List;
 import java.util.Map;
@@ -83,6 +84,9 @@ protected void createDiagnosisKeyDistributionMap(Collection<DiagnosisKey> diagno
           if (diagnosisKeyAccumulator.size() >= minNumberOfKeysPerBundle) {
             this.distributableDiagnosisKeys.put(currentHour, new ArrayList<>(diagnosisKeyAccumulator));
             diagnosisKeyAccumulator.clear();
+          } else {
+            // placeholder list is needed to be able to generate empty file - see issue #650
+            this.distributableDiagnosisKeys.put(currentHour, Collections.emptyList());
           }
         });
   }
diff --git a/services/distribution/src/main/java/app/coronawarn/server/services/distribution/config/DistributionServiceConfig.java b/services/distribution/src/main/java/app/coronawarn/server/services/distribution/config/DistributionServiceConfig.java
--- a/services/distribution/src/main/java/app/coronawarn/server/services/distribution/config/DistributionServiceConfig.java
+++ b/services/distribution/src/main/java/app/coronawarn/server/services/distribution/config/DistributionServiceConfig.java
@@ -464,6 +464,7 @@ public static class ObjectStore {
     @Min(1)
     @Max(64)
     private Integer maxNumberOfS3Threads;
+    private Boolean forceUpdateKeyfiles;
 
     public String getAccessKey() {
       return accessKey;
@@ -528,5 +529,13 @@ public Integer getMaxNumberOfS3Threads() {
     public void setMaxNumberOfS3Threads(Integer maxNumberOfS3Threads) {
       this.maxNumberOfS3Threads = maxNumberOfS3Threads;
     }
+
+    public Boolean getForceUpdateKeyfiles() {
+      return forceUpdateKeyfiles;
+    }
+
+    public void setForceUpdateKeyfiles(Boolean forceUpdateKeyfiles) {
+      this.forceUpdateKeyfiles = forceUpdateKeyfiles;
+    }
   }
 }
diff --git a/services/distribution/src/main/java/app/coronawarn/server/services/distribution/objectstore/S3Publisher.java b/services/distribution/src/main/java/app/coronawarn/server/services/distribution/objectstore/S3Publisher.java
--- a/services/distribution/src/main/java/app/coronawarn/server/services/distribution/objectstore/S3Publisher.java
+++ b/services/distribution/src/main/java/app/coronawarn/server/services/distribution/objectstore/S3Publisher.java
@@ -79,28 +79,25 @@ public S3Publisher(ObjectStoreAccess objectStoreAccess, FailedObjectStoreOperati
   }
 
   /**
-   * Synchronizes the files to S3.
+   * Synchronizes the files to S3. Current strategy is to never update diagnosis key archive files already
+   * published on S3, even if the retention and shifting policies cause a diff between subsequent distribution runs.
+   * Thus, by default distribution will only add new key files, but still modify indexes. This behaviour can however
+   * be controlled through the configuration parameter <code>DistributionServiceConfig.forceUpdateKeyFiles</code>
    *
    * @param root The path of the directory that shall be published.
+   * @see Github issue #650
    * @throws IOException in case there were problems reading files from the disk.
    */
   public void publish(Path root) throws IOException {
-    PublishedFileSet published;
     List<LocalFile> toPublish = new PublishFileSet(root).getFiles();
-    List<LocalFile> diff;
 
-    try {
-      published = new PublishedFileSet(
-          objectStoreAccess.getObjectsWithPrefix(distributionServiceConfig.getApi().getVersionPath()));
-      diff = toPublish
-          .stream()
-          .filter(published::isNotYetPublished)
-          .collect(Collectors.toList());
-    } catch (ObjectStoreOperationFailedException e) {
-      failedOperationsCounter.incrementAndCheckThreshold(e);
-      // failed to retrieve existing files; publish everything
-      diff = toPublish;
-    }
+    PublishedFileSet published = new PublishedFileSet(
+        objectStoreAccess.getObjectsWithPrefix(distributionServiceConfig.getApi().getVersionPath()),
+        distributionServiceConfig.getObjectStore().getForceUpdateKeyfiles());
+    List<LocalFile> diff = toPublish
+        .stream()
+        .filter(published::shouldPublish)
+        .collect(Collectors.toList());
 
     logger.info("Beginning upload of {} files... ", diff.size());
     try {
diff --git a/services/distribution/src/main/java/app/coronawarn/server/services/distribution/objectstore/client/S3Object.java b/services/distribution/src/main/java/app/coronawarn/server/services/distribution/objectstore/client/S3Object.java
--- a/services/distribution/src/main/java/app/coronawarn/server/services/distribution/objectstore/client/S3Object.java
+++ b/services/distribution/src/main/java/app/coronawarn/server/services/distribution/objectstore/client/S3Object.java
@@ -63,6 +63,15 @@ public String getCwaHash() {
     return cwaHash;
   }
 
+  /**
+   * Indicates if the S3 object is a file with diagnosis key content.
+   * The evaluation is based on the distribution logic which implies that such files are generated
+   * with a Date / Hour S3 key format (days: 1-31 / hours: 0-23) ending in 2 digits.
+   */
+  public boolean isDiagnosisKeyFile() {
+    return Objects.nonNull(objectName) && objectName.matches(".*\\d\\d");
+  }
+
   @Override
   public boolean equals(Object o) {
     if (this == o) {
diff --git a/services/distribution/src/main/java/app/coronawarn/server/services/distribution/objectstore/publish/LocalFile.java b/services/distribution/src/main/java/app/coronawarn/server/services/distribution/objectstore/publish/LocalFile.java
--- a/services/distribution/src/main/java/app/coronawarn/server/services/distribution/objectstore/publish/LocalFile.java
+++ b/services/distribution/src/main/java/app/coronawarn/server/services/distribution/objectstore/publish/LocalFile.java
@@ -97,11 +97,21 @@ public String getContentType() {
     if (s3Key.endsWith("app_config")) {
       return "application/zip";
     }
-    if (s3Key.matches(".*\\d")) {
+    if (isKeyFile()) {
       // date and hourly diagnosis key files
       return "application/zip";
     }
     // list of versions, dates, hours
     return "application/json";
   }
+
+  /**
+   * Indicates if a local file is a Key-file or not. Only the Key files are stored in the Date / Hour tree structure.
+   * One file per sub-folder (days: 1-31 / hours: 0-23). The index files are not stored in folders ending with a digit.
+   * 
+   * @return <code>true</code> if and only if the {@link #s3Key} ends with a digit, false otherwise.
+   */
+  public boolean isKeyFile() {
+    return s3Key.matches(".*\\d");
+  }
 }
diff --git a/services/distribution/src/main/java/app/coronawarn/server/services/distribution/objectstore/publish/PublishedFileSet.java b/services/distribution/src/main/java/app/coronawarn/server/services/distribution/objectstore/publish/PublishedFileSet.java
--- a/services/distribution/src/main/java/app/coronawarn/server/services/distribution/objectstore/publish/PublishedFileSet.java
+++ b/services/distribution/src/main/java/app/coronawarn/server/services/distribution/objectstore/publish/PublishedFileSet.java
@@ -32,37 +32,52 @@ public class PublishedFileSet {
 
   /** ta map of S3 objects with the S3 object name as the key component of the map. */
   private Map<String, S3Object> s3Objects;
+  private boolean isKeyFilePublishingAllowed;
+
 
   /**
-   * Creates a new PublishedFileSet for the given S3 objects with the help of the metadata provider.
-   * The metadata provider helps to determine whether files have been changed, and are requiring
-   * re-upload.
+   * Creates a new PublishedFileSet for the given S3 objects with the help of the
+   * metadata provider. The metadata provider helps to determine whether files
+   * have been changed, and are requiring re-upload.
    *
-   * @param s3Objects the list of s3 objects.
+   * @param s3Objects                  the list of s3 objects.
+   * @param isKeyFilePublishingAllowed whether the system is currently configured
+   *                                   to allow diagnosis key file updates on S3
    */
-  public PublishedFileSet(List<S3Object> s3Objects) {
+  public PublishedFileSet(List<S3Object> s3Objects, boolean isKeyFilePublishingAllowed) {
     this.s3Objects = s3Objects.stream()
         .collect(Collectors.toMap(S3Object::getObjectName, s3object -> s3object));
+    this.isKeyFilePublishingAllowed = isKeyFilePublishingAllowed;
   }
 
   /**
-   * Checks whether the given file, which is subject for publishing, is already available on the S3.
-   * Will return true, when:
+   * Checks whether the given file, which is subject for publishing, is already available on the S3. Will return true,
+   * when:
    * <ul>
-   *   <li>The S3 object key exists on S3</li>
-   *   <li>The checksum of the existing S3 object matches the hash of the given file</li>
+   *   <li>The S3 object key does NOT exist on S3.</li>
+   *   <li>The environment variable FORCE_UPDATE_KEYFILES is set to true.</li>
+   *   <li>The checksum of the existing S3 object differs to the hash of the given file.</li>
    * </ul>
    *
    * @param file the to-be-published file which should be checked
-   * @return true, if it exists and is identical
+   * @return {@code true}, if it doesn't exist or differs - {@code false}, if the file has been published already
    */
-  public boolean isNotYetPublished(LocalFile file) {
+  public boolean shouldPublish(LocalFile file) {
     S3Object published = s3Objects.get(file.getS3Key());
 
     if (published == null) {
       return true;
     }
 
+    if (file.isKeyFile()) {
+      // #650 - once published key files should not be changed anymore unless explicitly forced
+      return isKeyFilePublishingAllowed && contentChanged(file, published);
+    }
+
+    return contentChanged(file, published);
+  }
+
+  private boolean contentChanged(LocalFile file, S3Object published) {
     return !file.getChecksum().equals(published.getCwaHash());
   }
 
diff --git a/services/distribution/src/main/java/app/coronawarn/server/services/distribution/runner/RetentionPolicy.java b/services/distribution/src/main/java/app/coronawarn/server/services/distribution/runner/RetentionPolicy.java
--- a/services/distribution/src/main/java/app/coronawarn/server/services/distribution/runner/RetentionPolicy.java
+++ b/services/distribution/src/main/java/app/coronawarn/server/services/distribution/runner/RetentionPolicy.java
@@ -53,7 +53,7 @@ public class RetentionPolicy implements ApplicationRunner {
   /**
    * Creates a new RetentionPolicy.
    */
-  RetentionPolicy(DiagnosisKeyService diagnosisKeyService,
+  public RetentionPolicy(DiagnosisKeyService diagnosisKeyService,
       ApplicationContext applicationContext,
       DistributionServiceConfig distributionServiceConfig,
       S3RetentionPolicy s3RetentionPolicy) {
diff --git a/services/distribution/src/main/java/app/coronawarn/server/services/distribution/runner/S3Distribution.java b/services/distribution/src/main/java/app/coronawarn/server/services/distribution/runner/S3Distribution.java
--- a/services/distribution/src/main/java/app/coronawarn/server/services/distribution/runner/S3Distribution.java
+++ b/services/distribution/src/main/java/app/coronawarn/server/services/distribution/runner/S3Distribution.java
@@ -20,6 +20,7 @@
 
 package app.coronawarn.server.services.distribution.runner;
 
+import app.coronawarn.server.services.distribution.Application;
 import app.coronawarn.server.services.distribution.assembly.component.OutputDirectoryProvider;
 import app.coronawarn.server.services.distribution.objectstore.S3Publisher;
 import app.coronawarn.server.services.distribution.objectstore.client.ObjectStoreOperationFailedException;
@@ -29,6 +30,7 @@
 import org.slf4j.LoggerFactory;
 import org.springframework.boot.ApplicationArguments;
 import org.springframework.boot.ApplicationRunner;
+import org.springframework.context.ApplicationContext;
 import org.springframework.core.annotation.Order;
 import org.springframework.stereotype.Component;
 
@@ -43,10 +45,13 @@ public class S3Distribution implements ApplicationRunner {
 
   private final OutputDirectoryProvider outputDirectoryProvider;
   private final S3Publisher s3Publisher;
+  private final ApplicationContext applicationContext;
 
-  S3Distribution(OutputDirectoryProvider outputDirectoryProvider, S3Publisher s3Publisher) {
+  S3Distribution(OutputDirectoryProvider outputDirectoryProvider, S3Publisher s3Publisher,
+      ApplicationContext applicationContext) {
     this.outputDirectoryProvider = outputDirectoryProvider;
     this.s3Publisher = s3Publisher;
+    this.applicationContext = applicationContext;
   }
 
   @Override
@@ -58,6 +63,7 @@ public void run(ApplicationArguments args) {
       logger.info("Data pushed to Object Store successfully.");
     } catch (UnsupportedOperationException | ObjectStoreOperationFailedException | IOException e) {
       logger.error("Distribution failed.", e);
+      Application.killApplication(applicationContext);
     }
   }
 }
diff --git a/services/distribution/src/main/resources/application.yaml b/services/distribution/src/main/resources/application.yaml
--- a/services/distribution/src/main/resources/application.yaml
+++ b/services/distribution/src/main/resources/application.yaml
@@ -79,6 +79,8 @@ services:
       max-number-of-failed-operations: 5
       # The ThreadPoolTaskExecutor's maximum thread pool size.
       max-number-of-s3-threads: 4
+      # Allows distribution to overwrite files which are published on the object store
+      force-update-keyfiles: ${FORCE_UPDATE_KEYFILES:false}
 
 spring:
   main:

</patch>
diff --git a/services/distribution/src/test/java/app/coronawarn/server/services/distribution/assembly/diagnosiskeys/ProdDiagnosisKeyBundlerKeyRetrievalTest.java b/services/distribution/src/test/java/app/coronawarn/server/services/distribution/assembly/diagnosiskeys/ProdDiagnosisKeyBundlerKeyRetrievalTest.java
--- a/services/distribution/src/test/java/app/coronawarn/server/services/distribution/assembly/diagnosiskeys/ProdDiagnosisKeyBundlerKeyRetrievalTest.java
+++ b/services/distribution/src/test/java/app/coronawarn/server/services/distribution/assembly/diagnosiskeys/ProdDiagnosisKeyBundlerKeyRetrievalTest.java
@@ -89,7 +89,7 @@ void testGetsDatesWithDistributableDiagnosisKeys() {
   @MethodSource("createDiagnosisKeysForEpochDay0")
   void testGetDatesForEpochDay0(Collection<DiagnosisKey> diagnosisKeys) {
     bundler.setDiagnosisKeys(diagnosisKeys, LocalDateTime.of(1970, 1, 5, 0, 0));
-    var expDates = Set.of(LocalDate.ofEpochDay(2L));
+    var expDates = Set.of(LocalDate.ofEpochDay(2L), LocalDate.ofEpochDay(3L));
     var actDates = bundler.getDatesWithDistributableDiagnosisKeys();
     assertThat(actDates).isEqualTo(expDates);
   }
diff --git a/services/distribution/src/test/java/app/coronawarn/server/services/distribution/assembly/diagnosiskeys/structure/directory/DiagnosisKeysDateDirectoryTest.java b/services/distribution/src/test/java/app/coronawarn/server/services/distribution/assembly/diagnosiskeys/structure/directory/DiagnosisKeysDateDirectoryTest.java
--- a/services/distribution/src/test/java/app/coronawarn/server/services/distribution/assembly/diagnosiskeys/structure/directory/DiagnosisKeysDateDirectoryTest.java
+++ b/services/distribution/src/test/java/app/coronawarn/server/services/distribution/assembly/diagnosiskeys/structure/directory/DiagnosisKeysDateDirectoryTest.java
@@ -46,6 +46,7 @@
 import java.util.stream.IntStream;
 import org.junit.Rule;
 import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Disabled;
 import org.junit.jupiter.api.Test;
 import org.junit.jupiter.api.extension.ExtendWith;
 import org.junit.rules.TemporaryFolder;
@@ -105,9 +106,10 @@ void testCreatesCorrectDirectoryStructureForMultipleDates() {
     runDateDistribution(diagnosisKeys, LocalDateTime.of(1970, 1, 6, 0, 0));
     Set<String> actualFiles = Helpers.getFilePaths(outputFile, outputFile.getAbsolutePath());
     Set<String> expectedDateAndHourFiles = getExpectedDateAndHourFiles(Map.of(
-        "1970-01-03", List.of("0", "1", "2", "3", "4"),
-        "1970-01-04", List.of("0", "1", "2", "3", "4"),
-        "1970-01-05", List.of("0", "1", "2", "3", "4")), "1970-01-06");
+        "1970-01-03", listOfHoursAsStrings(0, 23),
+        "1970-01-04", listOfHoursAsStrings(0, 23),
+        "1970-01-05", listOfHoursAsStrings(0, 23)),
+    	"1970-01-06");
     assertThat(actualFiles).isEqualTo(expectedDateAndHourFiles);
   }
 
@@ -124,14 +126,17 @@ void testDoesNotIncludeCurrentDateInDirectoryStructure() {
     runDateDistribution(diagnosisKeys, LocalDateTime.of(1970, 1, 5, 12, 0));
     Set<String> actualFiles = Helpers.getFilePaths(outputFile, outputFile.getAbsolutePath());
     Set<String> expectedDateAndHourFiles = getExpectedDateAndHourFiles(Map.of(
-        "1970-01-03", List.of("0", "1", "2", "3", "4"),
-        "1970-01-04", List.of("0", "1", "2", "3", "4"),
-        "1970-01-05", List.of("0", "1", "2", "3", "4")), "1970-01-05");
+        "1970-01-03", listOfHoursAsStrings(0, 23),
+        "1970-01-04", listOfHoursAsStrings(0, 23),
+        "1970-01-05", listOfHoursAsStrings(0, 11)), "1970-01-05");
     assertThat(actualFiles).isEqualTo(expectedDateAndHourFiles);
   }
 
   @Test
-  void testDoesNotIncludeEmptyDatesInDirectoryStructure() {
+  @Disabled("Temporarily disabling this test as part of the fix for issue #650."
+  		+ "There seems to be a timing issue with this test because running it individually works, but running it"
+  		+ " in a suite will cause it to produce a different output then expected. Further investigation is required here ")
+  void testIncludesEmptyDatesInDirectoryStructure() {
     Collection<DiagnosisKey> diagnosisKeys = IntStream.range(0, 3)
         .filter(currentDate -> currentDate != 1)
         .mapToObj(currentDate -> IntStream.range(0, 5)
@@ -145,13 +150,14 @@ void testDoesNotIncludeEmptyDatesInDirectoryStructure() {
     runDateDistribution(diagnosisKeys, LocalDateTime.of(1970, 1, 5, 12, 0));
     Set<String> actualFiles = Helpers.getFilePaths(outputFile, outputFile.getAbsolutePath());
     Set<String> expectedDateAndHourFiles = getExpectedDateAndHourFiles(Map.of(
-        "1970-01-03", List.of("0", "1", "2", "3", "4"),
-        "1970-01-05", List.of("0", "1", "2", "3", "4")), "1970-01-05");
+        "1970-01-03", listOfHoursAsStrings(0, 23),
+        "1970-01-04", listOfHoursAsStrings(0, 23),
+        "1970-01-05", listOfHoursAsStrings(0, 11)), "1970-01-05");
     assertThat(actualFiles).isEqualTo(expectedDateAndHourFiles);
   }
 
   @Test
-  void testDoesNotIncludeDatesWithTooFewKeysInDirectoryStructure() {
+  void testIncludesDatesWithFewerKeysThanThresholdInDirectoryStructure() {
     Collection<DiagnosisKey> diagnosisKeys = List.of(
         buildDiagnosisKeys(6, LocalDateTime.of(1970, 1, 3, 1, 0), 5),
         buildDiagnosisKeys(6, LocalDateTime.of(1970, 1, 4, 1, 0), 4),
@@ -162,8 +168,10 @@ void testDoesNotIncludeDatesWithTooFewKeysInDirectoryStructure() {
     runDateDistribution(diagnosisKeys, LocalDateTime.of(1970, 1, 6, 12, 0));
     Set<String> actualFiles = Helpers.getFilePaths(outputFile, outputFile.getAbsolutePath());
     Set<String> expectedDateAndHourFiles = getExpectedDateAndHourFiles(Map.of(
-        "1970-01-03", List.of("1"),
-        "1970-01-05", List.of("1")), "1970-01-06");
+        "1970-01-03", listOfHoursAsStrings(1, 23),
+        "1970-01-04", listOfHoursAsStrings(0, 23),
+        "1970-01-05", listOfHoursAsStrings(0, 23),
+        "1970-01-06", listOfHoursAsStrings(0, 11)), "1970-01-06");
     assertThat(actualFiles).isEqualTo(expectedDateAndHourFiles);
   }
 
@@ -179,13 +187,16 @@ void testDoesNotIncludeDatesInTheFuture() {
     runDateDistribution(diagnosisKeys, LocalDateTime.of(1970, 1, 4, 12, 0));
     Set<String> actualFiles = Helpers.getFilePaths(outputFile, outputFile.getAbsolutePath());
     Set<String> expectedDateAndHourFiles = getExpectedDateAndHourFiles(Map.of(
-        "1970-01-03", List.of("1"),
-        "1970-01-04", List.of("1")), "1970-01-04");
+        "1970-01-03", listOfHoursAsStrings(1, 23),
+        "1970-01-04", listOfHoursAsStrings(0, 11)), "1970-01-04");
     assertThat(actualFiles).isEqualTo(expectedDateAndHourFiles);
   }
 
   @Test
   void testWhenDemoProfileIsActiveItDoesIncludeCurrentDateInDirectoryStructure() {
+    // set the incomplete days configuration for this particular test but revert before test ends
+    // such that other tests are
+    Boolean currentIncompleteDaysConfig = distributionServiceConfig.getIncludeIncompleteDays();
     distributionServiceConfig.setIncludeIncompleteDays(true);
     Collection<DiagnosisKey> diagnosisKeys = IntStream.range(0, 3)
         .mapToObj(currentDate -> IntStream.range(0, 5)
@@ -198,12 +209,19 @@ void testWhenDemoProfileIsActiveItDoesIncludeCurrentDateInDirectoryStructure() {
     runDateDistribution(diagnosisKeys, LocalDateTime.of(1970, 1, 5, 12, 0));
     Set<String> actualFiles = Helpers.getFilePaths(outputFile, outputFile.getAbsolutePath());
     Set<String> expectedDateAndHourFiles = getExpectedDateAndHourFiles(Map.of(
-        "1970-01-03", List.of("0", "1", "2", "3", "4"),
-        "1970-01-04", List.of("0", "1", "2", "3", "4"),
-        "1970-01-05", List.of("0", "1", "2", "3", "4")), "1970-01-05");
+        "1970-01-03", listOfHoursAsStrings(0, 23),
+        "1970-01-04", listOfHoursAsStrings(0, 23),
+        "1970-01-05", listOfHoursAsStrings(0, 11)), "1970-01-05");
     expectedDateAndHourFiles.addAll(Set.of(
         String.join(separator, "date", "1970-01-05", "index"),
         String.join(separator, "date", "1970-01-05", "index.checksum")));
+
+    distributionServiceConfig.setIncludeIncompleteDays(currentIncompleteDaysConfig);
     assertThat(actualFiles).isEqualTo(expectedDateAndHourFiles);
   }
+
+  private static List<String> listOfHoursAsStrings(int from, int until) {
+    return IntStream.range(from, until + 1).mapToObj(String::valueOf).collect(Collectors.toList());
+  }
+
 }
diff --git a/services/distribution/src/test/java/app/coronawarn/server/services/distribution/assembly/diagnosiskeys/structure/directory/DiagnosisKeysDirectoryTest.java b/services/distribution/src/test/java/app/coronawarn/server/services/distribution/assembly/diagnosiskeys/structure/directory/DiagnosisKeysDirectoryTest.java
--- a/services/distribution/src/test/java/app/coronawarn/server/services/distribution/assembly/diagnosiskeys/structure/directory/DiagnosisKeysDirectoryTest.java
+++ b/services/distribution/src/test/java/app/coronawarn/server/services/distribution/assembly/diagnosiskeys/structure/directory/DiagnosisKeysDirectoryTest.java
@@ -148,7 +148,8 @@ void checkBuildsTheCorrectDirectoryStructure() {
         join(s, "diagnosis-keys", "country", "DE", "date", "1970-01-03", "hour", "17", "index"),
         join(s, "diagnosis-keys", "country", "DE", "date", "1970-01-03", "hour", "18", "index"),
         join(s, "diagnosis-keys", "country", "DE", "date", "1970-01-03", "hour", "19", "index"),
-        // One missing
+        // One missing from data, but still we should a structure created because of the empty file (issue #650)
+        join(s, "diagnosis-keys", "country", "DE", "date", "1970-01-03", "hour", "20", "index"),
         join(s, "diagnosis-keys", "country", "DE", "date", "1970-01-03", "hour", "21", "index"),
         join(s, "diagnosis-keys", "country", "DE", "date", "1970-01-03", "hour", "22", "index"),
         join(s, "diagnosis-keys", "country", "DE", "date", "1970-01-03", "hour", "23", "index"),
@@ -159,7 +160,25 @@ void checkBuildsTheCorrectDirectoryStructure() {
         join(s, "diagnosis-keys", "country", "DE", "date", "1970-01-04", "hour", "2", "index"),
         join(s, "diagnosis-keys", "country", "DE", "date", "1970-01-04", "hour", "3", "index"),
         join(s, "diagnosis-keys", "country", "DE", "date", "1970-01-04", "hour", "4", "index"),
-        join(s, "diagnosis-keys", "country", "DE", "date", "1970-01-04", "hour", "5", "index")
+        join(s, "diagnosis-keys", "country", "DE", "date", "1970-01-04", "hour", "5", "index"),
+        join(s, "diagnosis-keys", "country", "DE", "date", "1970-01-04", "hour", "6", "index"),
+        join(s, "diagnosis-keys", "country", "DE", "date", "1970-01-04", "hour", "7", "index"),
+        join(s, "diagnosis-keys", "country", "DE", "date", "1970-01-04", "hour", "8", "index"),
+        join(s, "diagnosis-keys", "country", "DE", "date", "1970-01-04", "hour", "9", "index"),
+        join(s, "diagnosis-keys", "country", "DE", "date", "1970-01-04", "hour", "10", "index"),
+        join(s, "diagnosis-keys", "country", "DE", "date", "1970-01-04", "hour", "11", "index"),
+        join(s, "diagnosis-keys", "country", "DE", "date", "1970-01-04", "hour", "12", "index"),
+        join(s, "diagnosis-keys", "country", "DE", "date", "1970-01-04", "hour", "13", "index"),
+        join(s, "diagnosis-keys", "country", "DE", "date", "1970-01-04", "hour", "14", "index"),
+        join(s, "diagnosis-keys", "country", "DE", "date", "1970-01-04", "hour", "15", "index"),
+        join(s, "diagnosis-keys", "country", "DE", "date", "1970-01-04", "hour", "16", "index"),
+        join(s, "diagnosis-keys", "country", "DE", "date", "1970-01-04", "hour", "17", "index"),
+        join(s, "diagnosis-keys", "country", "DE", "date", "1970-01-04", "hour", "18", "index"),
+        join(s, "diagnosis-keys", "country", "DE", "date", "1970-01-04", "hour", "19", "index"),
+        join(s, "diagnosis-keys", "country", "DE", "date", "1970-01-04", "hour", "20", "index"),
+        join(s, "diagnosis-keys", "country", "DE", "date", "1970-01-04", "hour", "21", "index"),
+        join(s, "diagnosis-keys", "country", "DE", "date", "1970-01-04", "hour", "22", "index"),
+        join(s, "diagnosis-keys", "country", "DE", "date", "1970-01-04", "hour", "23", "index")
     );
 
     Set<String> actualFiles = getFilePaths(outputFile, outputFile.getAbsolutePath());
diff --git a/services/distribution/src/test/java/app/coronawarn/server/services/distribution/assembly/diagnosiskeys/structure/directory/DiagnosisKeysHourDirectoryTest.java b/services/distribution/src/test/java/app/coronawarn/server/services/distribution/assembly/diagnosiskeys/structure/directory/DiagnosisKeysHourDirectoryTest.java
--- a/services/distribution/src/test/java/app/coronawarn/server/services/distribution/assembly/diagnosiskeys/structure/directory/DiagnosisKeysHourDirectoryTest.java
+++ b/services/distribution/src/test/java/app/coronawarn/server/services/distribution/assembly/diagnosiskeys/structure/directory/DiagnosisKeysHourDirectoryTest.java
@@ -44,6 +44,7 @@
 import java.util.Set;
 import java.util.stream.Collectors;
 import java.util.stream.IntStream;
+
 import org.junit.Rule;
 import org.junit.jupiter.api.BeforeEach;
 import org.junit.jupiter.api.Test;
@@ -100,23 +101,11 @@ void testCreatesCorrectStructureForMultipleHours() {
         .mapToObj(currentHour -> buildDiagnosisKeys(6, LocalDateTime.of(1970, 1, 3, 0, 0).plusHours(currentHour), 5))
         .flatMap(List::stream)
         .collect(Collectors.toList());
-    runHourDistribution(diagnosisKeys, LocalDateTime.of(1970, 1, 5, 0, 0),
+    runHourDistribution(diagnosisKeys, LocalDateTime.of(1970, 1, 4, 0, 0),
         LocalDate.of(1970, 1, 3));
     Set<String> actualFiles = getFilePaths(outputFile, outputFile.getAbsolutePath());
-    assertThat(actualFiles).isEqualTo(getExpectedHourFiles(Set.of("0", "1", "2", "3", "4")));
-  }
-
-  @Test
-  void testDoesNotIncludeEmptyHours() {
-    Collection<DiagnosisKey> diagnosisKeys = IntStream.range(0, 5)
-        .filter(currentHour -> currentHour != 3)
-        .mapToObj(currentHour -> buildDiagnosisKeys(6, LocalDateTime.of(1970, 1, 3, 0, 0).plusHours(currentHour), 5))
-        .flatMap(List::stream)
-        .collect(Collectors.toList());
-    runHourDistribution(diagnosisKeys, LocalDateTime.of(1970, 1, 5, 0, 0),
-        LocalDate.of(1970, 1, 3));
-    Set<String> actualFiles = getFilePaths(outputFile, outputFile.getAbsolutePath());
-    assertThat(actualFiles).isEqualTo(getExpectedHourFiles(Set.of("0", "1", "2", "4")));
+    assertThat(actualFiles).isEqualTo(getExpectedHourFiles(
+        IntStream.range(0, 24).mapToObj(String::valueOf).collect(Collectors.toSet())));
   }
 
   @Test
@@ -131,21 +120,6 @@ void testDoesNotIncludeCurrentHour() {
     assertThat(actualFiles).isEqualTo(getExpectedHourFiles(Set.of("0", "1", "2", "3")));
   }
 
-  @Test
-  void testDoesNotIncludeHoursWithTooFewKeys() {
-    Collection<DiagnosisKey> diagnosisKeys = List.of(
-        buildDiagnosisKeys(6, LocalDateTime.of(1970, 1, 3, 0, 0), 5),
-        buildDiagnosisKeys(6, LocalDateTime.of(1970, 1, 3, 1, 0), 4),
-        buildDiagnosisKeys(6, LocalDateTime.of(1970, 1, 3, 2, 0), 5))
-        .stream()
-        .flatMap(List::stream)
-        .collect(Collectors.toList());
-    runHourDistribution(diagnosisKeys, LocalDateTime.of(1970, 1, 6, 12, 0),
-        LocalDate.of(1970, 1, 3));
-    Set<String> actualFiles = getFilePaths(outputFile, outputFile.getAbsolutePath());
-    assertThat(actualFiles).isEqualTo(getExpectedHourFiles(Set.of("0", "2")));
-  }
-
   @Test
   void testDoesNotIncludeHoursInTheFuture() {
     Collection<DiagnosisKey> diagnosisKeys = List.of(
diff --git a/services/distribution/src/test/java/app/coronawarn/server/services/distribution/assembly/diagnosiskeys/structure/directory/decorator/DateIndexingDecoratorTest.java b/services/distribution/src/test/java/app/coronawarn/server/services/distribution/assembly/diagnosiskeys/structure/directory/decorator/DateIndexingDecoratorTest.java
--- a/services/distribution/src/test/java/app/coronawarn/server/services/distribution/assembly/diagnosiskeys/structure/directory/decorator/DateIndexingDecoratorTest.java
+++ b/services/distribution/src/test/java/app/coronawarn/server/services/distribution/assembly/diagnosiskeys/structure/directory/decorator/DateIndexingDecoratorTest.java
@@ -66,24 +66,6 @@ void setup() {
     diagnosisKeyBundler = new ProdDiagnosisKeyBundler(distributionServiceConfig);
   }
 
-  @Test
-  void excludesEmptyDatesFromIndex() {
-    List<DiagnosisKey> diagnosisKeys = Stream
-        .of(buildDiagnosisKeys(6, LocalDateTime.of(1970, 1, 3, 0, 0), 5),
-            buildDiagnosisKeys(6, LocalDateTime.of(1970, 1, 4, 0, 0), 0),
-            buildDiagnosisKeys(6, LocalDateTime.of(1970, 1, 5, 0, 0), 5))
-        .flatMap(List::stream)
-        .collect(Collectors.toList());
-    diagnosisKeyBundler.setDiagnosisKeys(diagnosisKeys, LocalDateTime.of(1970, 1, 6, 0, 0));
-    DateIndexingDecorator decorator = makeDecoratedDateDirectory(diagnosisKeyBundler);
-    decorator.prepare(new ImmutableStack<>().push("DE"));
-
-    Set<LocalDate> index = decorator.getIndex(new ImmutableStack<>());
-
-    assertThat(index).contains(LocalDate.of(1970, 1, 3))
-        .doesNotContain(LocalDate.of(1970, 1, 4))
-        .contains(LocalDate.of(1970, 1, 5));
-  }
 
   @Test
   void excludesCurrentDateFromIndex() {
@@ -112,7 +94,7 @@ void excludesDatesThatExceedTheMaximumNumberOfKeys() {
     when(svcConfig.getMaximumNumberOfKeysPerBundle()).thenReturn(1);
 
     DiagnosisKeyBundler diagnosisKeyBundler = new ProdDiagnosisKeyBundler(svcConfig);
-    diagnosisKeyBundler.setDiagnosisKeys(diagnosisKeys, LocalDateTime.of(1970, 1, 5, 0, 0));
+    diagnosisKeyBundler.setDiagnosisKeys(diagnosisKeys, LocalDateTime.of(1970, 1, 4, 0, 0));
 
     DateIndexingDecorator decorator = makeDecoratedDateDirectory(diagnosisKeyBundler);
 
diff --git a/services/distribution/src/test/java/app/coronawarn/server/services/distribution/assembly/diagnosiskeys/structure/directory/decorator/HourIndexingDecoratorTest.java b/services/distribution/src/test/java/app/coronawarn/server/services/distribution/assembly/diagnosiskeys/structure/directory/decorator/HourIndexingDecoratorTest.java
--- a/services/distribution/src/test/java/app/coronawarn/server/services/distribution/assembly/diagnosiskeys/structure/directory/decorator/HourIndexingDecoratorTest.java
+++ b/services/distribution/src/test/java/app/coronawarn/server/services/distribution/assembly/diagnosiskeys/structure/directory/decorator/HourIndexingDecoratorTest.java
@@ -73,7 +73,7 @@ void tearDown() {
   }
 
   @Test
-  void excludesHoursThatExceedTheMaximumNumberOfKeys() {
+  void excludesHoursThatExceedTheMaximumNumberOfKeysPerBundle() {
     List<DiagnosisKey> diagnosisKeys = buildDiagnosisKeys(6, LocalDateTime.of(1970, 1, 3, 4, 0), 2);
 
     TimeUtils.setNow(LocalDateTime.of(1970, 1, 3, 0, 0).toInstant(ZoneOffset.UTC));
@@ -84,7 +84,7 @@ void excludesHoursThatExceedTheMaximumNumberOfKeys() {
     when(svcConfig.getMaximumNumberOfKeysPerBundle()).thenReturn(1);
 
     DiagnosisKeyBundler diagnosisKeyBundler = new ProdDiagnosisKeyBundler(svcConfig);
-    diagnosisKeyBundler.setDiagnosisKeys(diagnosisKeys, LocalDateTime.of(1970, 1, 5, 0, 0));
+    diagnosisKeyBundler.setDiagnosisKeys(diagnosisKeys, LocalDateTime.of(1970, 1, 3, 5, 0));
 
     HourIndexingDecorator decorator = makeDecoratedHourDirectory(diagnosisKeyBundler);
 
diff --git a/services/distribution/src/test/java/app/coronawarn/server/services/distribution/common/DiagnosisTestData.java b/services/distribution/src/test/java/app/coronawarn/server/services/distribution/common/DiagnosisTestData.java
new file mode 100644
--- /dev/null
+++ b/services/distribution/src/test/java/app/coronawarn/server/services/distribution/common/DiagnosisTestData.java
@@ -0,0 +1,87 @@
+/*-
+ * ---license-start
+ * Corona-Warn-App
+ * ---
+ * Copyright (C) 2020 SAP SE and all other contributors
+ * ---
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ * ---license-end
+ */
+
+package app.coronawarn.server.services.distribution.common;
+
+import app.coronawarn.server.common.persistence.domain.DiagnosisKey;
+import java.time.LocalDate;
+import java.time.LocalDateTime;
+import java.time.ZoneOffset;
+import java.time.temporal.ChronoUnit;
+import java.util.List;
+import java.util.Random;
+import java.util.stream.Collectors;
+import java.util.stream.IntStream;
+
+/**
+ * A simple container class able to create diagnosis keys test data, given a set of days or day intervals expresed in
+ * <code>java.time.*</code> structures.
+ */
+public final class DiagnosisTestData {
+
+  private List<DiagnosisKey> diagnosisKeys;
+
+  private DiagnosisTestData() {
+  }
+
+  public List<DiagnosisKey> getDiagnosisKeys() {
+    return diagnosisKeys;
+  }
+
+  /**
+   * @return An instance that contains diagnosis keys computed for the given interval of days. Each day will have a
+   * number of keys equal to the given <code>casesPerDay</code> parameter, recorded, for simplicity, in the last hour of
+   * the day.
+   */
+  public static DiagnosisTestData of(LocalDate fromDay, LocalDate untilDay, int casesPerDay) {
+
+    int numberOfDays = (int) ChronoUnit.DAYS.between(fromDay, untilDay) + 1;
+    DiagnosisTestData testData = new DiagnosisTestData();
+
+    testData.diagnosisKeys = computeDiagnosisKeys(testData, fromDay, numberOfDays, casesPerDay);
+    return testData;
+  }
+
+  private static List<DiagnosisKey> computeDiagnosisKeys(DiagnosisTestData testData, LocalDate fromDay,
+      int numberOFDays, int casesPerDay) {
+    return IntStream.range(0, numberOFDays)
+        .mapToObj(day ->
+            randomDiagnosisKeys(fromDay.atStartOfDay().plusDays(day).plusHours(23), casesPerDay)
+        ).flatMap(List::stream).collect(Collectors.toList());
+  }
+
+  private static List<DiagnosisKey> randomDiagnosisKeys(LocalDateTime submissionTime, int casesPerDay) {
+    long timestamp = submissionTime.toEpochSecond(ZoneOffset.UTC) / 3600;
+    return IntStream.range(0, casesPerDay)
+        .mapToObj(dayCounter ->
+            DiagnosisKey.builder().withKeyData(randomKeyData())
+                .withRollingStartIntervalNumber(600)
+                .withTransmissionRiskLevel(5).withSubmissionTimestamp(timestamp)
+                .build())
+        .collect(Collectors.toList());
+  }
+
+  private static byte[] randomKeyData() {
+    byte[] exposureKeys = new byte[16];
+    Random random = new Random();
+    random.nextBytes(exposureKeys);
+    return exposureKeys;
+  }
+}
diff --git a/services/distribution/src/test/java/app/coronawarn/server/services/distribution/objectstore/S3PublisherTest.java b/services/distribution/src/test/java/app/coronawarn/server/services/distribution/objectstore/S3PublisherTest.java
--- a/services/distribution/src/test/java/app/coronawarn/server/services/distribution/objectstore/S3PublisherTest.java
+++ b/services/distribution/src/test/java/app/coronawarn/server/services/distribution/objectstore/S3PublisherTest.java
@@ -126,15 +126,6 @@ void uploadOneDueToOneChanged() throws IOException {
     verify(objectStoreAccess, times(1)).putObject(any());
   }
 
-  @Test
-  void uploadAllDueToError() throws IOException {
-    when(objectStoreAccess.getObjectsWithPrefix("version")).thenThrow(new ObjectStoreOperationFailedException(""));
-
-    s3Publisher.publish(publishingPath);
-
-    verify(objectStoreAccess, times(3)).putObject(any());
-  }
-
   @Test
   void executorGetsShutDown() throws IOException {
     when(objectStoreAccess.getObjectsWithPrefix("version")).thenReturn(emptyList());
diff --git a/services/distribution/src/test/java/app/coronawarn/server/services/distribution/objectstore/client/S3ClientWrapperTest.java b/services/distribution/src/test/java/app/coronawarn/server/services/distribution/objectstore/client/S3ClientWrapperTest.java
--- a/services/distribution/src/test/java/app/coronawarn/server/services/distribution/objectstore/client/S3ClientWrapperTest.java
+++ b/services/distribution/src/test/java/app/coronawarn/server/services/distribution/objectstore/client/S3ClientWrapperTest.java
@@ -48,6 +48,7 @@
 import org.junit.jupiter.params.provider.ValueSource;
 import org.springframework.beans.factory.annotation.Autowired;
 import org.springframework.beans.factory.annotation.Value;
+import org.springframework.boot.autoconfigure.condition.ConditionalOnMissingBean;
 import org.springframework.boot.context.properties.EnableConfigurationProperties;
 import org.springframework.boot.test.context.ConfigFileApplicationContextInitializer;
 import org.springframework.boot.test.mock.mockito.MockBean;
@@ -98,7 +99,8 @@ class S3ClientWrapperTest {
   public static class RetryS3ClientConfig {
 
     @Bean
-    public ObjectStoreClient createS3ClientWrapper(S3Client s3Client) {
+    @ConditionalOnMissingBean
+    public ObjectStoreClient createObjectStoreClient(S3Client s3Client) {
       return new S3ClientWrapper(s3Client);
     }
   }
@@ -215,7 +217,7 @@ void testPutObjectForNoHeaders() {
   @Test
   void testPutObjectForContentTypeHeader() {
     String contentType = "foo-content-type";
-    s3ClientWrapper.putObject(VALID_BUCKET_NAME, VALID_NAME, Path.of(""), 
+    s3ClientWrapper.putObject(VALID_BUCKET_NAME, VALID_NAME, Path.of(""),
         newHashMap(HeaderKey.CONTENT_TYPE, contentType));
 
     PutObjectRequest expRequest =
diff --git a/services/distribution/src/test/java/app/coronawarn/server/services/distribution/objectstore/ObjectStoreAccessIT.java b/services/distribution/src/test/java/app/coronawarn/server/services/distribution/objectstore/integration/ObjectStoreAccessIT.java
similarity index 96%
rename from services/distribution/src/test/java/app/coronawarn/server/services/distribution/objectstore/ObjectStoreAccessIT.java
rename to services/distribution/src/test/java/app/coronawarn/server/services/distribution/objectstore/integration/ObjectStoreAccessIT.java
--- a/services/distribution/src/test/java/app/coronawarn/server/services/distribution/objectstore/ObjectStoreAccessIT.java
+++ b/services/distribution/src/test/java/app/coronawarn/server/services/distribution/objectstore/integration/ObjectStoreAccessIT.java
@@ -18,13 +18,14 @@
  * ---license-end
  */
 
-package app.coronawarn.server.services.distribution.objectstore;
+package app.coronawarn.server.services.distribution.objectstore.integration;
 
 import static org.assertj.core.api.Assertions.assertThat;
 import static org.mockito.Mockito.spy;
 import static org.mockito.Mockito.when;
 
 import app.coronawarn.server.services.distribution.config.DistributionServiceConfig;
+import app.coronawarn.server.services.distribution.objectstore.ObjectStoreAccess;
 import app.coronawarn.server.services.distribution.objectstore.client.ObjectStorePublishingConfig;
 import app.coronawarn.server.services.distribution.objectstore.client.S3Object;
 import app.coronawarn.server.services.distribution.objectstore.publish.LocalFile;
diff --git a/services/distribution/src/test/java/app/coronawarn/server/services/distribution/objectstore/integration/ObjectStoreFilePreservationIT.java b/services/distribution/src/test/java/app/coronawarn/server/services/distribution/objectstore/integration/ObjectStoreFilePreservationIT.java
new file mode 100644
--- /dev/null
+++ b/services/distribution/src/test/java/app/coronawarn/server/services/distribution/objectstore/integration/ObjectStoreFilePreservationIT.java
@@ -0,0 +1,214 @@
+/*-
+ * ---license-start
+ * Corona-Warn-App
+ * ---
+ * Copyright (C) 2020 SAP SE and all other contributors
+ * ---
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ * ---license-end
+ */
+
+package app.coronawarn.server.services.distribution.objectstore.integration;
+
+import java.io.File;
+import java.io.IOException;
+import java.time.LocalDate;
+import java.time.Month;
+import java.time.temporal.ChronoUnit;
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
+
+import org.junit.Rule;
+import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Tag;
+import org.junit.jupiter.api.Test;
+import org.junit.jupiter.api.extension.ExtendWith;
+import org.junit.rules.TemporaryFolder;
+import org.mockito.Mockito;
+import org.springframework.beans.factory.annotation.Autowired;
+import org.springframework.boot.test.context.ConfigFileApplicationContextInitializer;
+import org.springframework.boot.test.mock.mockito.MockBean;
+import org.springframework.context.ApplicationContext;
+import org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor;
+import org.springframework.test.annotation.DirtiesContext;
+import org.springframework.test.context.ActiveProfiles;
+import org.springframework.test.context.ContextConfiguration;
+import org.springframework.test.context.junit.jupiter.SpringExtension;
+
+import app.coronawarn.server.common.persistence.service.DiagnosisKeyService;
+import app.coronawarn.server.services.distribution.Application;
+import app.coronawarn.server.services.distribution.assembly.component.OutputDirectoryProvider;
+import app.coronawarn.server.services.distribution.assembly.structure.directory.DirectoryOnDisk;
+import app.coronawarn.server.services.distribution.common.DiagnosisTestData;
+import app.coronawarn.server.services.distribution.config.DistributionServiceConfig;
+import app.coronawarn.server.services.distribution.objectstore.FailedObjectStoreOperationsCounter;
+import app.coronawarn.server.services.distribution.objectstore.ObjectStoreAccess;
+import app.coronawarn.server.services.distribution.objectstore.S3Publisher;
+import app.coronawarn.server.services.distribution.objectstore.S3RetentionPolicy;
+import app.coronawarn.server.services.distribution.objectstore.client.S3Object;
+import app.coronawarn.server.services.distribution.runner.Assembly;
+import app.coronawarn.server.services.distribution.runner.RetentionPolicy;
+
+@ExtendWith(SpringExtension.class)
+@ContextConfiguration(classes = Application.class, initializers = ConfigFileApplicationContextInitializer.class)
+@DirtiesContext
+@ActiveProfiles("integration-test")
+@Tag("s3-integration")
+class ObjectStoreFilePreservationIT {
+
+  @Autowired
+  private DiagnosisKeyService diagnosisKeyService;
+  @Autowired
+  private Assembly fileAssembler;
+  @Autowired
+  private ApplicationContext applicationContext;
+  @Autowired
+  private S3RetentionPolicy s3RetentionPolicy;
+  @Autowired
+  private ObjectStoreAccess objectStoreAccess;
+  @Autowired
+  private DistributionServiceConfig distributionServiceConfig;
+
+  @MockBean
+  private OutputDirectoryProvider distributionDirectoryProvider;
+
+  @Rule
+  private TemporaryFolder testOutputFolder = new TemporaryFolder();
+
+
+  @BeforeEach
+  public void setup() throws IOException {
+    testOutputFolder.create();
+    objectStoreAccess.deleteObjectsWithPrefix(distributionServiceConfig.getApi().getVersionPath());
+  }
+
+  /**
+   * The test covers a behaviour that manifests itself when data retention and shifting policies cause the file
+   * distribution logic to generate, in subsequent runs, different content for the same timeframes. Below the
+   * distribution problem is described with a concerete daily scenario.
+   * <p>
+   * The test presumes there are 4 consecutive days with 80 keys submitted daily. Running a distribution in Day 4 would
+   * result in:
+   * <p>
+   *     Day 1   -> submission of 80 keys   -> 1 empty file distributed<br>
+   *     Day 2   -> submission of 80 keys   -> 1 distributed containing 160 keys<br>
+   *     Day 3   -> submission of 80 keys   -> 1 empty file distributed<br>
+   *     Day 4   -> submission of 80 keys   -> 1 distributed containing 160 keys<br>
+   * <p>
+   * All day & hour files already generated should not be changed/removed from S3 even after retention policies have
+   * been applied and a second distribution is triggered.
+   * <p>
+   * If for example, data in Day 1 gets removed completely, then a second distribution run causes a shifting of keys in
+   * different files compared to the previous run, the result being:
+   *  <p>
+   *     Day 2   -> submission of 80 keys   -> 1 empty file generated (different than what is currently on S3)<br>
+   *     Day 3   -> submission of 80 keys   -> 1 distributed containing 160 keys (different than 1 empty file on S3)<br>
+   *     Day 4   -> submission of 80 keys   -> 1 empty file (different than 1 empty file on S3)<br>
+   */
+  @Test
+  void files_once_published_to_objectstore_should_not_be_overriden_because_of_retention_or_shifting_policies()
+      throws IOException {
+
+    // keep data in the past for this test
+    LocalDate testStartDate = LocalDate.now().minusDays(10);
+    LocalDate testEndDate = LocalDate.now().minusDays(6);
+
+    // setup the 80 keys per day scenario
+    createDiagnosisKeyTestData(testStartDate, testEndDate, 80);
+
+    assembleAndDistribute(testOutputFolder.newFolder("output-before-retention"));
+    List<S3Object> filesBeforeRetention = getPublishedFiles();
+
+    triggerRetentionPolicy(testStartDate);
+
+    // Trigger second distrubution after data retention policies were applied
+    assembleAndDistribute(testOutputFolder.newFolder("output-after-retention"));
+    List<S3Object> filesAfterRetention = getPublishedFiles();
+
+    assertPreviouslyPublishedKeyFilesAreTheSame(filesBeforeRetention, filesAfterRetention);
+  }
+
+  private List<S3Object> getPublishedFiles() {
+    return objectStoreAccess.getObjectsWithPrefix(distributionServiceConfig.getApi().getVersionPath());
+  }
+
+  private void assertPreviouslyPublishedKeyFilesAreTheSame(List<S3Object> filesBeforeRetention,
+      List<S3Object> filesAfterRetention) {
+
+    Map<String, S3Object> beforeRetentionFileMap = filesBeforeRetention
+        .stream()
+        .filter(S3Object::isDiagnosisKeyFile)
+        .collect(Collectors.toMap(S3Object::getObjectName, s3object -> s3object));
+
+    filesAfterRetention
+        .stream()
+        .filter(S3Object::isDiagnosisKeyFile)
+        .forEach(secondVersion -> {
+          S3Object previouslyPublished = beforeRetentionFileMap.get(secondVersion.getObjectName());
+
+          if (filesAreDifferent(previouslyPublished, secondVersion)) {
+            throw new AssertionError("Files have been changed on object store "
+                + "due to retention policy. Before: " + previouslyPublished.getObjectName()
+                + "-" + previouslyPublished.getCwaHash()
+                + "| After:" + secondVersion.getObjectName()
+                + "-" + secondVersion.getCwaHash());
+          }
+        });
+  }
+
+  private boolean filesAreDifferent(S3Object previouslyPublished, S3Object newVerion) {
+    return previouslyPublished == null ||
+        !newVerion.getCwaHash().equals(previouslyPublished.getCwaHash());
+  }
+
+  /**
+   * Remove test data inserted for the given date
+   */
+  private void triggerRetentionPolicy(LocalDate fromDate) {
+    DistributionServiceConfig mockDistributionConfig = new DistributionServiceConfig();
+    mockDistributionConfig.setRetentionDays(numberOfDaysSince(fromDate));
+    new RetentionPolicy(diagnosisKeyService, applicationContext, mockDistributionConfig,
+        s3RetentionPolicy).run(null);
+  }
+
+  private Integer numberOfDaysSince(LocalDate testStartDate) {
+    return (int) ChronoUnit.DAYS.between(testStartDate, LocalDate.now()) - 1;
+  }
+
+  private void createDiagnosisKeyTestData(LocalDate fromDay, LocalDate untilDay, int casesPerDay) {
+    DiagnosisTestData testData = DiagnosisTestData.of(fromDay, untilDay, casesPerDay);
+    diagnosisKeyService.saveDiagnosisKeys(testData.getDiagnosisKeys());
+  }
+
+  private void assembleAndDistribute(File output) throws IOException {
+    Mockito.when(distributionDirectoryProvider.getDirectory()).thenReturn(new DirectoryOnDisk(output));
+    Mockito.when(distributionDirectoryProvider.getFileOnDisk()).thenReturn(output);
+
+    fileAssembler.run(null);
+
+    S3Publisher s3Publisher = new S3Publisher(objectStoreAccess,
+        new FailedObjectStoreOperationsCounter(distributionServiceConfig),
+        newAsyncExecutor(), distributionServiceConfig);
+    s3Publisher.publish(distributionDirectoryProvider.getFileOnDisk().toPath().toAbsolutePath());
+  }
+
+  private ThreadPoolTaskExecutor newAsyncExecutor() {
+    ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
+    executor.setCorePoolSize(8);
+    executor.setMaxPoolSize(8);
+    executor.setQueueCapacity(11);
+    executor.initialize();
+    return executor;
+  }
+}
diff --git a/services/distribution/src/test/java/app/coronawarn/server/services/distribution/objectstore/S3PublisherIT.java b/services/distribution/src/test/java/app/coronawarn/server/services/distribution/objectstore/integration/S3PublisherIT.java
similarity index 89%
rename from services/distribution/src/test/java/app/coronawarn/server/services/distribution/objectstore/S3PublisherIT.java
rename to services/distribution/src/test/java/app/coronawarn/server/services/distribution/objectstore/integration/S3PublisherIT.java
--- a/services/distribution/src/test/java/app/coronawarn/server/services/distribution/objectstore/S3PublisherIT.java
+++ b/services/distribution/src/test/java/app/coronawarn/server/services/distribution/objectstore/integration/S3PublisherIT.java
@@ -18,11 +18,14 @@
  * ---license-end
  */
 
-package app.coronawarn.server.services.distribution.objectstore;
+package app.coronawarn.server.services.distribution.objectstore.integration;
 
 import static org.assertj.core.api.Assertions.assertThat;
 
 import app.coronawarn.server.services.distribution.config.DistributionServiceConfig;
+import app.coronawarn.server.services.distribution.objectstore.FailedObjectStoreOperationsCounter;
+import app.coronawarn.server.services.distribution.objectstore.ObjectStoreAccess;
+import app.coronawarn.server.services.distribution.objectstore.S3Publisher;
 import app.coronawarn.server.services.distribution.objectstore.client.ObjectStorePublishingConfig;
 import app.coronawarn.server.services.distribution.objectstore.client.S3Object;
 import java.io.IOException;
diff --git a/services/distribution/src/test/java/app/coronawarn/server/services/distribution/objectstore/publish/LocalFileTest.java b/services/distribution/src/test/java/app/coronawarn/server/services/distribution/objectstore/publish/LocalFileTest.java
--- a/services/distribution/src/test/java/app/coronawarn/server/services/distribution/objectstore/publish/LocalFileTest.java
+++ b/services/distribution/src/test/java/app/coronawarn/server/services/distribution/objectstore/publish/LocalFileTest.java
@@ -20,6 +20,8 @@
 
 package app.coronawarn.server.services.distribution.objectstore.publish;
 
+import static org.junit.Assert.assertFalse;
+import static org.junit.Assert.assertTrue;
 import static org.junit.jupiter.api.Assertions.assertEquals;
 
 import java.nio.file.Path;
@@ -48,4 +50,26 @@ void testGetContentTypeZip(String path) {
     LocalFile test = new LocalIndexFile(Path.of("/root", path, "/index"), Path.of("/root"));
     assertEquals("application/zip", test.getContentType());
   }
+
+  @ParameterizedTest
+  @ValueSource(strings = { 
+      "version/v1/diagnosis-keys/country/DE/date/2020-06-11",
+      "version/v1/diagnosis-keys/country/DE/date/2020-06-11/hour/13" })
+  void testIsKeyFile(String path) {
+    LocalFile test = new LocalIndexFile(Path.of("/root", path, "/index"), Path.of("/root"));
+    assertTrue(test.isKeyFile());
+  }
+
+  @ParameterizedTest
+  @ValueSource(strings = { 
+      "version",
+      "version/v1/configuration/country",
+      "version/v1/configuration/country/DE/app_config", 
+      "version/v1/diagnosis-keys/country",
+      "version/v1/diagnosis-keys/country/DE/date", 
+      "version/v1/diagnosis-keys/country/DE/date/2020-06-11/hour" })
+  void testIsNotKeyFile(String path) {
+    LocalFile test = new LocalIndexFile(Path.of("/root", path, "/index"), Path.of("/root"));
+    assertFalse(test.isKeyFile());
+  }
 }
diff --git a/services/distribution/src/test/java/app/coronawarn/server/services/distribution/objectstore/publish/PublishedFileSetTest.java b/services/distribution/src/test/java/app/coronawarn/server/services/distribution/objectstore/publish/PublishedFileSetTest.java
new file mode 100644
--- /dev/null
+++ b/services/distribution/src/test/java/app/coronawarn/server/services/distribution/objectstore/publish/PublishedFileSetTest.java
@@ -0,0 +1,72 @@
+/*-
+ * ---license-start
+ * Corona-Warn-App
+ * ---
+ * Copyright (C) 2020 SAP SE and all other contributors
+ * ---
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ * ---license-end
+ */
+
+package app.coronawarn.server.services.distribution.objectstore.publish;
+
+import static org.junit.Assert.assertFalse;
+import static org.junit.Assert.assertTrue;
+
+import java.nio.file.Path;
+import java.util.Collections;
+import java.util.List;
+
+import org.junit.jupiter.params.ParameterizedTest;
+import org.junit.jupiter.params.provider.ValueSource;
+
+import app.coronawarn.server.services.distribution.objectstore.client.S3Object;
+
+class PublishedFileSetTest {
+
+  @ParameterizedTest
+  @ValueSource(strings = {
+      "version/v1/diagnosis-keys/country/DE/date/2020-01-01",
+      "version/v1/diagnosis-keys/country/DE/date/2020-06-11/hour/0",
+      "version/v1/diagnosis-keys/country/DE/date/2020-06-11/hour/23"})
+  void testShouldNotPublishWithoutForceUpdateConfiguration(String key) {
+    List<S3Object> s3Objects = List.of(new S3Object(key, "1234"));
+    PublishedFileSet publishedSet = new PublishedFileSet(s3Objects,  false);
+    LocalFile testFile = new LocalIndexFile(Path.of("/root", key, "/index"), Path.of("/root"));
+    assertFalse(publishedSet.shouldPublish(testFile));
+  }
+
+  @ParameterizedTest
+  @ValueSource(strings = {
+      "version/v1/diagnosis-keys/country/DE/date/2020-01-01",
+      "version/v1/diagnosis-keys/country/DE/date/2020-06-11/hour/0",
+      "version/v1/diagnosis-keys/country/DE/date/2020-06-11/hour/23"})
+  void testShouldPublishWithForceUpdateConfiguration(String key) {
+    List<S3Object> s3Objects = List.of(new S3Object(key, "1234"));
+    PublishedFileSet publishedSet = new PublishedFileSet(s3Objects, true);
+    LocalFile testFile = new LocalIndexFile(Path.of("/root", key, "/index"), Path.of("/root"));
+    assertTrue(publishedSet.shouldPublish(testFile));
+  }
+
+  @ParameterizedTest
+  @ValueSource(strings = {
+      "version/v1/diagnosis-keys/country/DE/date/2020-01-01",
+      "version/v1/diagnosis-keys/country/DE/date/2020-06-11/hour/0",
+      "version/v1/diagnosis-keys/country/DE/date/2020-06-11/hour/23"})
+  void testShouldPublishWhenObjectStoreEmpty(String key) {
+    PublishedFileSet publishedSet = new PublishedFileSet(Collections.emptyList(), false);
+    LocalFile testFile = new LocalIndexFile(Path.of("/root", key, "/index"), Path.of("/root"));
+    assertTrue(publishedSet.shouldPublish(testFile));
+  }
+
+}
diff --git a/services/distribution/src/test/java/app/coronawarn/server/services/distribution/objectstore/publish/S3ObjectTest.java b/services/distribution/src/test/java/app/coronawarn/server/services/distribution/objectstore/publish/S3ObjectTest.java
new file mode 100644
--- /dev/null
+++ b/services/distribution/src/test/java/app/coronawarn/server/services/distribution/objectstore/publish/S3ObjectTest.java
@@ -0,0 +1,52 @@
+/*-
+ * ---license-start
+ * Corona-Warn-App
+ * ---
+ * Copyright (C) 2020 SAP SE and all other contributors
+ * ---
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ * ---license-end
+ */
+
+package app.coronawarn.server.services.distribution.objectstore.publish;
+
+import static org.junit.Assert.assertFalse;
+import static org.junit.Assert.assertTrue;
+
+import org.junit.jupiter.params.ParameterizedTest;
+import org.junit.jupiter.params.provider.ValueSource;
+
+import app.coronawarn.server.services.distribution.objectstore.client.S3Object;
+
+class S3ObjectTest {
+
+  @ParameterizedTest
+  @ValueSource(strings = {
+      "version/v1/diagnosis-keys/country/DE/date/2020-06-11",
+      "version/v1/diagnosis-keys/country/DE/date/2020-06-11/hour/13" })
+  void testIsKeyFile(String key) {
+    S3Object test = new S3Object(key);
+    assertTrue(test.isDiagnosisKeyFile());
+  }
+
+  @ParameterizedTest
+  @ValueSource(strings = {
+      "version/v1/configuration/country/DE/app_config",
+      "version/v1/configuration/country",
+      "version/v1/diagnosis-keys/country/DE/date",
+      "version/v1/diagnosis-keys/country/DE/date/2020-06-11/hour" })
+  void testIsNotKeyFile(String key) {
+    S3Object test = new S3Object(key);
+    assertFalse(test.isDiagnosisKeyFile());
+  }
+}
diff --git a/services/distribution/src/test/resources/application-integration-test.yaml b/services/distribution/src/test/resources/application-integration-test.yaml
new file mode 100644
--- /dev/null
+++ b/services/distribution/src/test/resources/application-integration-test.yaml
@@ -0,0 +1,4 @@
+---
+services:
+  distribution:
+    shifting-policy-threshold: 140
diff --git a/services/distribution/src/test/resources/application.yaml b/services/distribution/src/test/resources/application.yaml
--- a/services/distribution/src/test/resources/application.yaml
+++ b/services/distribution/src/test/resources/application.yaml
@@ -51,6 +51,7 @@ services:
       retry-backoff: 1
       max-number-of-failed-operations: 5
       max-number-of-s3-threads: 2
+      force-update-keyfiles: ${FORCE_UPDATE_KEYFILES:false}
 spring:
   main:
     banner-mode: off
