A Kafka topic exists for each hotfolder. It receives a message when a new file is uploaded.
Description
Hotfolders are used to upload files to cenTaur. The folders are constantly monitored by the system for new files. For each hotfolder, there is also a topic in Kafka. If there is a new file, a message is published to the respective hotfolder topic. Other microservices, Node-RED nodes, or business logic function can subscribe to a certain hotfolder topic and react on new files.
Message
Field description
Fieldname | Description |
|---|---|
| messageType | Type of message, for example "ImportFileMessage". |
| version | The version of this message format. |
| timestamp | Timestamp when this message was published to the message broker. |
| filesystem | Name of the filesystem (hotfolder) to which the files are uploaded. |
| files | Array with the list of files that are uploaded to the hotfolder. In most cases, it contains only one file. There are use cases where more than one file is uploaded. For example, the content of a directory. |
| files/objectName | Name of the file in the filesystem. The name can include path separators ("/"). |
| files/isDir | True if this file is a directory. |
| files/key | Key to reference the file. For example, if you call the File service to get a download URL. |
| files/url | Signed URL to download this file. This URL is valid for one week. The period starts with the value in the timestamp field. |
| lastModified | Last modification timestamp of the file set. If the set contains exactly one file, this is the last modification date of that file. In case the set contains multiple files, it is the latest of all modification dates. |
Example
{
"messageType":"ImportFileMessage",
"version":1,
"timestamp":1612967718944,
"filesystem":"hotfolder-test",
"files":
[
{
"objectName":"Abstract 1.jpg",
"isDir":false,
"key":"aG90Zm9sZGVyLXdlcm5lci9maWxlOkFic3RyYWN0IDEuanBnLyJlZ
WFhZWU0NWViMDAxODk5NjVhZjY2ZGQwYWFlMTkyNSI=",
"url":"https://minio-docu.censhare.com/hotfolder-test/
Abstract%201.jpg?X-Amz-Algorithm=AWS4-HMAC-SHA256&
X-Amz-Date=20210210T143518Z&X-Amz-SignedHeaders=
host&X-Amz-Expires=604800&X-Amz-Credential=
W3HJV23GCE8PS9LK7118%2F20210210%2Feu-central-1
%2Fs3%2Faws4_request&X-Amz-Signature=a80e46fff2702c
9bb74ec66db294b27a7f198252c7b356cd85972c2bbf5a70e3"
}
],
"lastModified":1612967715545
}Configuration
A hotfolder topic is created with the following values:
| Parameter | Value |
|---|---|
| Partitions | 1 |
| Replicas | 1 |