Difference between revisions of "OS4X fetch queue"
Line 14: | Line 14: | ||
= Configuration = | = Configuration = | ||
+ | OS4X fetch queue items have several general parameters and some are protocol specific. | ||
+ | |||
+ | The generic configuration options are: | ||
+ | *Active: if active, the entry will be picked by the fetch queue daemon. | ||
+ | *Verbose protocol output: If enabled, the complete protocol input and output will be logged in the fetch queue logs and contained in the dry-run window. | ||
+ | *Description: Making it easier for identification, each fetch queue entry contains a (non-unique) description, which will also be contained in the fetch queue logs. | ||
+ | *Server name or IP address: the server to which a connection will be made on the given TCP port. | ||
+ | *Username & password: self-explaining, the username and password for authentication. | ||
+ | *Server path: The target path on the server to fetch files from. If unsure use "<code>/</code>". | ||
+ | *Protocol: The selectable protocol. | ||
+ | *Traget folder prefix for downloaded entities: The target folder for downloaded files. If empty (by default) it will be a subdirectory of [[OS4X_Core_configuration#data_incoming_directory|OS4X's incoming directory]], followed by "<code>os4x-fetchqueue-</code>", the Unix timestamp, followed by a dash ("<code>-</code>") and the actual process ID | ||
+ | |||
+ | |||
+ | The protocol specific configuration options are: | ||
+ | *FTP(S): | ||
+ | **Disable EPVS: passive file transmission will be disabled ("ACTIVE" only") | ||
+ | **Disable EPRT: Disable the use of the EPRT and LPRT commands when doing active FTP transfers. | ||
+ | **Disable proxy: no FTP proxy will be used. | ||
+ | **Force HTTP 1.0 for proxy communication: If a FTP proxy is being used, it will be communication via (obsoleted) HTTP 1.0 only. For ancient environments only, you should really upgrade your environment. | ||
+ | |||
+ | |||
+ | |||
+ | [[File:Bildschirmfoto 2025-09-09 um 16.28.28.png]] | ||
= Dry-run = | = Dry-run = |
Revision as of 14:39, 9 September 2025
With OS4X fetch queue, you are able to automatically collect files from remote servers via various protocols. Feel free to ask for extended protocol support by submitting your request to support@os4x.com.
Required license
OS4X fetch queue comes in different license models:
- Stand-alone license (only OS4X fetch queue is licensed)
- Add-on license for OS4X Core
- Included in OS4X Enterprise
The OS4X fetch queue is visible in:
- Queues -> Fetch queue
The logs are visible via:
- Logs -> Fetch queue log
Configuration
OS4X fetch queue items have several general parameters and some are protocol specific.
The generic configuration options are:
- Active: if active, the entry will be picked by the fetch queue daemon.
- Verbose protocol output: If enabled, the complete protocol input and output will be logged in the fetch queue logs and contained in the dry-run window.
- Description: Making it easier for identification, each fetch queue entry contains a (non-unique) description, which will also be contained in the fetch queue logs.
- Server name or IP address: the server to which a connection will be made on the given TCP port.
- Username & password: self-explaining, the username and password for authentication.
- Server path: The target path on the server to fetch files from. If unsure use "
/
". - Protocol: The selectable protocol.
- Traget folder prefix for downloaded entities: The target folder for downloaded files. If empty (by default) it will be a subdirectory of OS4X's incoming directory, followed by "
os4x-fetchqueue-
", the Unix timestamp, followed by a dash ("-
") and the actual process ID
The protocol specific configuration options are:
- FTP(S):
- Disable EPVS: passive file transmission will be disabled ("ACTIVE" only")
- Disable EPRT: Disable the use of the EPRT and LPRT commands when doing active FTP transfers.
- Disable proxy: no FTP proxy will be used.
- Force HTTP 1.0 for proxy communication: If a FTP proxy is being used, it will be communication via (obsoleted) HTTP 1.0 only. For ancient environments only, you should really upgrade your environment.
Dry-run
When starting a dry-run of an OS4X fetch queue entry, the server is being connected to and the directory is listed for potential files which may be fetched. The files are being examined for regular expression and age information. If these tests succeed, an output will be printed out which displays what will be done afterwards. No file will be transferred during this process, also not deleted. If verbose output is enabled, all protocol information are being contained in the output window.
Logging
The logs in "Logs" -> "Fetch queue log" contain all information about fetching processes. If an OS4X Enterprise job is being created, it is directly viewable via the "info" icon on the left.
Daemon
The background daemon running OS4X fetch queue is "os4xfqd
", which is installed in OS4X's binary directory.. The daemon can be started and stopped from the top-left icon list:
Alternatively, you can manage the process at the panel "Programs" -> "Daemons":
Supported protocols
Starting at 2025-08-20, OS4X fetch queue supports the following protocols:
- FTP
- Implicit FTPS
- Explicit FTPS (via STARTTLS)
- SFTP
- WebDAV (via http)
- WebDAV (via https)
Why no SCP? - Well, the SCP protocol doesn't support a "delete
" operation, so fetched files cannot be deleted from the source server, making it impossible to distinguish between already fetched files and new ones.