OS4X fetch queue

With OS4X fetch queue, you are able to automatically collect files from remote servers via various protocols. Feel free to ask for extended protocol support by submitting your request to support@os4x.com.

Required license

OS4X fetch queue comes in different license models:

• Stand-alone license (only OS4X fetch queue is licensed)
• Add-on license for OS4X Core
• Included in OS4X Enterprise

Navigation

The OS4X fetch queue is visible in:

• Queues -> Fetch queue

The logs are visible via:

• Logs -> Fetch queue log

Configuration

OS4X fetch queue items have several general parameters and some are protocol specific.

Generic configuration options

• Active: if active, the entry will be picked by the fetch queue daemon.
• Verbose protocol output: If enabled, the complete protocol input and output will be logged in the fetch queue logs and contained in the dry-run window.
• Description: Making it easier for identification, each fetch queue entry contains a (non-unique) description, which will also be contained in the fetch queue logs.
• Server name or IP address: the server to which a connection will be made on the given TCP port.
• Username & password: self-explaining, the username and password for authentication.
• Server path: The target path on the server to fetch files from. If unsure use "/".
• Protocol: The selectable protocol.
• Target folder prefix for downloaded entities: The target folder for downloaded files. If empty (by default) it will be a subdirectory of OS4X's incoming directory, followed by "os4x-fetchqueue-", the Unix timestamp, followed by a dash ("-") and the actual process ID
• Regular expression matching files: For picking exactly the files you want, you must configure the regular expression (via the configured regular expression engine "POSIX" or the more powerful "PCRE")
• Ignore files with suffix ".part": Many upload tools append a filename suffix ".part" during the transmission and remove it afterwards. These files can be ignored by enabling this checkbox. This is also a handy feature if serialization of incoming OFTP2 files is enabled.
• Age of entry (seconds; 0: disable check): The minimum age of files. Be aware that some protocols may display times not correctly or the server uses a different time calculation.
• Position: The position of the fetch queue entry in the fetch queue entry list.
• Sleep time between found results (seconds): If required, transmission of files can be slowed done by defining an amount of seconds to wait between two file downloads.
• Ignore empty files: If enabled, files with a filesize of 0 bytes will be ignored.
• Type: Various things may happen after the transmission of a file. This can be, based on the license:
  • OS4X Enterprise receive job
  • OS4X Enterprise send job
  • OS4X Core
  • nothing (for a stand-alone license of OS4X FetchQ).
• Interval: The fetch queue entry will be executed every defined timeframe. The execution time will not be added to the time of the next execution, so if for example an interval is defined for 25 seconds, and the execution of the last run takes longer, the next fetch queue daemon run will execute the same rule again.
• Next run date: If defined, the next execution date can be configured.

Protocol specific configuration options

• FTP(S), explicit FTPS:
  • Disable EPVS: passive file transmission will be disabled ("ACTIVE" only")
  • Disable EPRT: Disable the use of the EPRT and LPRT commands when doing active FTP transfers.
  • Disable proxy: no FTP proxy will be used.
  • Force HTTP 1.0 for proxy communication: If a FTP proxy is being used, it will be communication via (obsoleted) HTTP 1.0 only. For ancient environments only, you should really upgrade your environment.
• SFTP:
  • Key file: instead of a password authentication, a SFTP key file (beware the permissions!) can be used.
• WebDAV:
  • none

Dry-run

When starting a dry-run of an OS4X fetch queue entry, the server is being connected to and the directory is listed for potential files which may be fetched. The files are being examined for regular expression and age information. If these tests succeed, an output will be printed out which displays what will be done afterwards. No file will be transferred during this process, also not deleted. If verbose output is enabled, all protocol information are being contained in the output window.

Logging

The logs in "Logs" -> "Fetch queue log" contain all information about fetching processes. If an OS4X Enterprise job is being created, it is directly viewable via the "info" icon on the left.

Daemon

The background daemon running OS4X fetch queue is "os4xfqd", which is installed in OS4X's binary directory.. The daemon can be started and stopped from the top-left icon list:

Alternatively, you can manage the process at the panel "Programs" -> "Daemons":

Supported protocols

Starting at 2025-08-20, OS4X fetch queue supports the following protocols:

• FTP
• Implicit FTPS
• Explicit FTPS (via STARTTLS)
• SFTP
• WebDAV (via http)
• WebDAV (via https)

Why no SCP? - Well, the SCP protocol doesn't support a "delete" operation, so fetched files cannot be deleted from the source server, making it impossible to distinguish between already fetched files and new ones.