Batch (sFTP) APIs
Version
Current Specification Version: V8.2.2
Supported File Formats
Processing
Tokenization Concept
General Information
The purpose of this document is to provide documentation for 3rd parties to integrate with TidyPay processing gateway. It includes information on basic gateway operations, folder structure, file name convention, file formats and tokenization.
The batch transaction processing is implemented through sFTP protocol. Batches are submitted as pipe (|) delimited files (one transaction per row). The protocol supports 2 operations: sale and credit and can accept both credit card and direct debit/eCheck transactions. Any sFTP enabled client can communicate with TidyPay; file upload/download can be done manually or using timer based automated process. To learn more about end-points used for file submission, review Security Management guide.
The batch transaction processing is implemented through sFTP protocol. Batches are submitted as pipe (|) delimited files (one transaction per row). The protocol supports 2 operations: sale and credit and can accept both credit card and direct debit/eCheck transactions. Any sFTP enabled client can communicate with TidyPay; file upload/download can be done manually or using timer based automated process. To learn more about end-points used for file submission, review Security Management guide.
Supported File Formats
Processing
To be able to process batch files, each portfolio, reseller and merchant within the gateway is provided with certain folders on the FTP server, which are used for uploading and downloading of various batch files.
Each portfolio folder is located in the root of the FTP server. Reseller folders are located under a folder of an associated portfolio. Merchant folders are located under a folder of an associated reseller. If no reseller is assigned to the merhant, its folder is created under a folder of an associated portfolio. This approach allows making access for batch processing more convenient: FTP user that corresponds to the service user of a reseller is created with an access to the folder of the corresponding reseller. Therefore, when a new merchant is created, the reseller should not re-configure an associated FTP user to grant access to the merchant's folder.
Below you can find detailed information about name convention of each folder type and the structure of available sub-folders.
Example:
Each portfolio folder is located in the root of the FTP server. Reseller folders are located under a folder of an associated portfolio. Merchant folders are located under a folder of an associated reseller. If no reseller is assigned to the merhant, its folder is created under a folder of an associated portfolio. This approach allows making access for batch processing more convenient: FTP user that corresponds to the service user of a reseller is created with an access to the folder of the corresponding reseller. Therefore, when a new merchant is created, the reseller should not re-configure an associated FTP user to grant access to the merchant's folder.
Below you can find detailed information about name convention of each folder type and the structure of available sub-folders.
Portfolio Folders
The name of a portfolio folder is comprised of two parts: letter p and the ID of the portfolio. Each portfolio folder may contain the following sub-folders:- inbox - a folder for uploading files to be processed;
- outbox - a folder that contains returns, CAU returns, subscription returns, charges, charge payments files, aggregated merchant statements;
- data - a folder for files exchange.
Example:
Portfolio ID: 100 Inbox: ftps://[server-name]/p100/inbox Outbox: ftps://[server-name]/p100/outbox Data: ftps://[server-name]/p100/data
Reseller Folders
The name of a reseller folder is comprised of two parts: letter r and the ID of the reseller. Each reseller folder may contain the following sub-folders:
- inbox - a folder for uploading files to be processed;
- outbox - a folder that contains returns, CAU returns, subscription returns, charges, charge payments files;
- data - a folder for files exchange.
Example:
Reseller ID: 3
Inbox: ftps://[server-name]/p100/r3/inbox Outbox: ftps://[server-name]/p100/r3/outbox Data: ftps://[server-name]/p100/r3/data
Merchant Folders
The name of a merchant folder corresponds the ID of the merchant. Each merchant folder contains the following sub-folders:
- inbox (required) - a folder for uploading files to be processed;
- outbox (required) - a folder that contains processing results;
- invault - a folder for uploading files for tokenization prior to processing. The folder is used when a batch file contains raw account numbers. In this case, all files are required to be submitted to the invault folder to be tokenized. After being tokenized, the files are subsequently put into the inbox folder for regular processing. See integration notes for more information regarding files that contain unencrypted account numbers.
- data - a folder for files exchange.
Be advised that sFTP folders are not created automatically. So if you assign new reseller to a merchant, you have to create new folders for this merchant manually.
Example:
Merchant ID: 2000 Inbox: ftps://[server-name]/p100/r3/2000/inbox Invault: ftps://[server-name]/p100/r3/2000/invault Outbox: ftps://[server-name]/p100/r3/2000/outbox Data: ftps://[server-name]/p100/r3/2000/data
File Format
All files, uploaded to the FTP, are required to be submitted as a single pipe-delimited text file in CSV format. To reduce the size of the file and ensure its integrity, the file should be archived by any standard ZIP client with no password protection applied. The resulting ZIP file has to be placed in /inbox folder for processing. For cases when the user interface is used for file uploading, a file format can be either CSV or ZIP.If a zipped file contains any sensitive information, it can be additionally encrypted using PGP algorithm. Files are encrypted with a public key that you can request from the gateway support team. If the file is encrypted, .pgp is added after file extension. To learn more about batch file encryption, see integration notes.
After a file is uploaded to the FTP server, it gets processed by the gateway and submitted to a processor. When the response is received from the processor, it is imported to the gateway and a corresponding response file is generated. The response file, which is a ZIP archive with a compressed CSV file inside, can be downloaded from the /outbox folder. The response has the exact same name as the request file.
If the request file is damaged, the system generates Parsing Error File. If the file contains validation errors, the system generates Validation Error File. Both files have CSV format and are placed in /outbox folder after being generated. Note that if an error file is generated, your request file is rejected, transactions are not processed and a response file is not received.
File Name Convention
File names consist of the following parts:- name of the file that can be anything that integrator desires. Can be a combination of letters, digits and dots. It is recommended to include a timestamp in yyyyMMdd or yyyyMMddHHmmss format as part of this name (for example, 20150930);
- suffix that corresponds to the type of a file (for example .returns, .au, .capture, etc);
- file extension (for example .csv, .zip, .zip.pgp).
Names of the ZIP file and included CSV file are required to match. Examples of the names can be reviewed in the table below.
| Submission Date and Time (optional): | Sept 30th 2015 at 5:46 AM |
| Request File | |
| Request CSV (pipe) File Name: | monthly_billing_201509300546.csv |
| Request ZIP File Name: | monthly_billing_201509300546.zip |
| Request PGP File Name: | monthly_billing_201509300546.zip.pgp |
| Response File | |
| Response CSV (pipe) File Name: | monthly_billing_201509300546.csv |
| Response ZIP File Name: | monthly_billing_201509300546.zip |
| Response PGP File Name: | monthly_billing_201509300546.zip.pgp |
| Parsing File | |
| Parsing Error File Name: | monthly_billing_201509300546.error.csv |
| Validation Error File | |
| Validation Error File Name: | monthly_billing_201509300546.validation.csv |
| Account Update File | |
| Account Update CSV (pipe) File Name: | account_update_20150930.2001.au.csv |
| Account Update ZIP File Name (by account): | account_update_20150930.2001.au.zip |
| Account Update PGP File Name: | account_update_20150930.2001.au.zip.pgp |
| Subscription File | |
| Subscription CSV (pipe) File Name (by account): | subscription_20150930.2001.subscriptions.csv |
| Subscription ZIP File Name (by account): | subscription_20150930.2001.subscriptions.zip |
| Subscription PGP File Name (by account): | subscription_20150930.2001.subscriptions.zip.pgp |
| Returns File | |
| Returns CSV (pipe) File Name (by merchant): | 20150930.1.2000.returns.csv |
| Returns CSV (pipe) File Name (by account): | 20150930.1.2001.returns.csv |
| Returns ZIP File Name (by merchant): | 20150930.1.2000.returns.zip |
| Returns ZIP File Name (by account): | 20150930.1.2001.returns.zip |
| Returns Account Update File | |
| Returns Account Update CSV (pipe) File Name (by merchant): | 20150930.2000.returns.au.csv |
| Returns Account Update CSV (pipe) File Name (by account): | 20150930.2001.returns.au.csv |
| Returns Account Update ZIP File Name (by merchant): | 20150930.2000.returns.au.zip |
| Returns Account Update ZIP File Name (by account): | 20150930.2001.returns.au.zip |
| Returns Subscription File | |
| Returns Subscription CSV (pipe) File Name (by merchant): | 20150930.2000.returns.subscriptions.csv |
| Returns Subscription CSV (pipe) File Name (by account): | 20150930.2001.returns.subscriptions.csv |
| Returns Subscription ZIP File Name (by merchant): | 20150930.2000.returns.subscriptions.zip |
| Returns Subscription ZIP File Name (by account): | 20150930.2001.returns.subscriptions.zip |
| Capture File | |
| Capture CSV (pipe) File Name: | 20150930.capture.csv |
| Capture ZIP File Name: | 20150930.capture.zip |
| Capture PGP File Name: | 20150930.capture.zip.pgp |
| Charges File | |
| Charges CSV (pipe) File Name: | 20150930.charges.csv |
| Charges ZIP File Name: | 20150930.charges.zip |
| Charges PGP File Name: | 20150930.charges.zip.pgp |
| Charge Payments File | |
| Charge Payments CSV (pipe) File Name: | 20150930.payments.csv |
| Charge Payments ZIP File Name: | 20150930.payments.zip |
| Charge Payments PGP File Name: | 20150930.payments.zip.pgp |
Processing request file contains a list of transactions that need to be processed.
Returns, chargebacks and chargeback reversals, as well as results of rebill process (declines or approvals), can be received in two ways based on merchant's selection:
To review the format of the file, see Processing File Format.
Request File
To submit transactions for processing, you have to submit a corresponding request file. To review the naming convention of the file, see integration notes.Response File
After a request file is submitted to a processor and the respective response is received, a response file is generated. The process differs for payment card and direct debit transactions:- For payment card transactions, an outcome of the transaction is available in most cases in real-time. Therefore, the response included in the file reflects the final outcome of the transaction. A processed file can't be canceled.
- For direct debit transactions, a situation is more complicated because most direct debit systems are not working in real-time. Therefore, all transactions are marked by the gateway as approved when being sent to a processor, and there is a certain period, during which returns file, containing transactions that were not processed, can be generated. A processed file can't be canceled.
Returns, chargebacks and chargeback reversals, as well as results of rebill process (declines or approvals), can be received in two ways based on merchant's selection:
- As part of the response file - returns, chargebacks, reversals and rebill results will be accumulated by the system and included in the standard response file when it is generated;
- As a separate file - returns, chargebacks, reversals and rebill results will be placed in a separate file, which will be generated on daily basis. The file will not be generated if there are no transactions of these types for a given day. The file is placed in /outbox folder.
To review the format of the file, see Processing File Format.
Tokenization File Format
Tokenization request files are required to be submitted as a single pipe-delimited text file in CSV format. To reduce the size of the file and ensure its integrity, the file should be archived by any standard ZIP client with no password protection applied. The resulting ZIP file has to be placed in /invault folder for processing.After a file is uploaded to the FTP server, it gets tokenized, processed by the gateway and submitted to a processor. When the response is received from the processor, it is imported to the gateway and a corresponding response file is generated. The response file can be downloaded from the /outbox folder. Response files are ZIP archives with compressed CSV files inside. Name of the response file corresponds to the name of request file.
When the request file is damaged and cannot be read or processed, a parsing error file is generated. The file contains a message that explains why the file could not be processed.
To review the format of the file, see Tokenization File Format.
Tokenization File Name Convention
File names consist of the following parts:- name of the file that can be a combination of letters, digits and dots;
- suffix that corresponds to the type of a file (.tokens or .tokens.error);
- file extension (.csv or .zip).
Names of the ZIP file and included CSV file are required to match. Examples of the names can be reviewed in the table below.
| Tokenization Request ZIP File Name: | account_numbers.tokens.zip |
| Tokenization Request CSV (pipe) File Name: | account_numbers.tokens.csv |
| Tokenization Response ZIP File Name: | account_numbers.tokens.zip |
| Tokenization Response CSV (pipe) File Name: | account_numbers.tokens.csv |
| Parsing Error File Name: | account_numbers.tokens.error.csv |
Parsing error file is generated when a request file is damaged and cannot be read or processed. For example, the file is generated when it is impossible to extract a ZIP file or read the content of the file. The file contains a message that explains why the file could not be processed. If an error file has been received, the entire request file must be re-uploaded for re-processing.
To review the naming convention of the file, see integration notes. To review the format of the file, see Parsing Error File Format.
To review the naming convention of the file, see integration notes. To review the format of the file, see Parsing Error File Format.
Validation error file is generated when a request file includes some records that cannot be processed due to validation errors. The file contains a list of transactions that could not be parsed or properly persisted, with errorCode and errorMessage values associated with every transaction and explaining the specific issue with it. To review the naming convention of the file, see integration notes.
Critical validation errors are:
If a request file contains non-critical validation errors, validation file is not generated. In this case, all transactions, except those that got non-critical validation errors, will be processed by a processor. Non-critical validation errors are the following:
In some cases, a submitter that sends a request file for processing prefers the file to be either processed completely or not processed at all if it includes any errors. To control whether a validation file should be generated, the following setting can be configured on the user interface:
Critical validation errors are:
- Invalid characters – non-numeric symbols are added into numeric fields.
- Invalid date format – date format differs from yyyyMMdd for transactionDate field and from mmYY for the accountAccessory field.
- Invalid token – token doesn’t follow required token structure.
- Invalid account ID – incorrect value for the accountId field.
If a request file contains non-critical validation errors, validation file is not generated. In this case, all transactions, except those that got non-critical validation errors, will be processed by a processor. Non-critical validation errors are the following:
- Incorrect card number – card number is not validated by mod 10 algorithm verification.
- Invalid routing number – routing number is not well-formatted.
- Required fields are missing.
In some cases, a submitter that sends a request file for processing prefers the file to be either processed completely or not processed at all if it includes any errors. To control whether a validation file should be generated, the following setting can be configured on the user interface:
- All Or Fail - if there is at least one invalid transaction in the request file, validation error file is generated and the request file is not processed at all.
- Any Valid - all valid transactions included in a request file are processed by a processor. Invalid transactions are placed into the validation error file and should be re-processed as a separate file after correcting all of the issues.
Account update file contains a list of payment card transactions, for which updated data need to be received. Processors do not imply any timeline requirements for the account data to be updated. It should be noted that when CAU is used, the merchant is charged a fee for each transaction checked by the processor.
CAU processing runs similar to the processing of the request/response files. However, processing of an account update file on processor’s side may not start immediately after submission. It can start in 48 hours after submission or earlier depending on the time when account update transactions were submitted to the system.
For CAU, aggregation is optional for all processors except Vantiv Tandem. If aggregation is used, the gateway submits the account update data for multiple merchants in a single file. In this case, you should enable aggregation within a provider profile that the account update is performed through. If the processor receives an aggregated file, it sends an aggregated response for all merchants submitted in the request.
To submit account update transactions to the system, you have to use a corresponding file. To review the naming convention of the file, see integration notes. To review the format of the file, see Account Update File Format.
CAU processing runs similar to the processing of the request/response files. However, processing of an account update file on processor’s side may not start immediately after submission. It can start in 48 hours after submission or earlier depending on the time when account update transactions were submitted to the system.
For CAU, aggregation is optional for all processors except Vantiv Tandem. If aggregation is used, the gateway submits the account update data for multiple merchants in a single file. In this case, you should enable aggregation within a provider profile that the account update is performed through. If the processor receives an aggregated file, it sends an aggregated response for all merchants submitted in the request.
To submit account update transactions to the system, you have to use a corresponding file. To review the naming convention of the file, see integration notes. To review the format of the file, see Account Update File Format.
Subscription file is a file that contains a list of customers' bank accounts that need to be registered with the bank/processor for further transaction processing (for a mandate-based system, such as BACS or SEPA). See integration notes for more information.
To submit subscriptions (mandates) to the system, you have to use a corresponding file. To review the naming convention of the file, see integration notes. To review the format of the file, see Subscription File Format.
To submit subscriptions (mandates) to the system, you have to use a corresponding file. To review the naming convention of the file, see integration notes. To review the format of the file, see Subscription File Format.
Returns file contains direct debit returns, chargebacks/reversals and results of the automated retry process (approvals/declines that occurred during the day).
By default, it is generated once per day. When a processor supports delivery of returns files several times per day, the gateway provides the ability to obtain an additional returns file. In this case, a merchant receives two returns files per day that are generated at 11AM and 8PM server time.
To review the naming convention of the file, see integration notes. To review the format of the file, see Returns File Format.
By default, it is generated once per day. When a processor supports delivery of returns files several times per day, the gateway provides the ability to obtain an additional returns file. In this case, a merchant receives two returns files per day that are generated at 11AM and 8PM server time.
To review the naming convention of the file, see integration notes. To review the format of the file, see Returns File Format.
Returns CAU file contains a list of payment cards that got updated. It is generated once per day if any data from a processor is available. Note that this file format is available for American Express only.
To review the naming convention of the file, see integration notes. To review the format of the file, see Returns Account Update File Format.
To review the naming convention of the file, see integration notes. To review the format of the file, see Returns Account Update File Format.
Returns subscription file contains subscriptions (mandates) that had been initially approved but were terminated by a processor/bank after some period of time. It is generated once per day if one or more initially approved subscriptions get declined.
To review the naming convention of the file, see integration notes. To review the format of the file, see Returns Subscription File Format.
To review the naming convention of the file, see integration notes. To review the format of the file, see Returns Subscription File Format.
Capture file contains a list of real-time sale-auth transactions that are going to be captured. A format of the file allows a submitter to perform either full or partial capture.
To submit sale-auth/credit-auth transactions to the system, you have to use a capture file. Note that a prior configuration has to be set via the user interface. To allow for batch capture to be enabled, make sure that Batch Capture Enabled checkbox is activated in the provider profile of the processor that you use for real-time processing.
To review the naming convention of the file, see integration notes. To review the format of the file, see Capture File Format.
To submit sale-auth/credit-auth transactions to the system, you have to use a capture file. Note that a prior configuration has to be set via the user interface. To allow for batch capture to be enabled, make sure that Batch Capture Enabled checkbox is activated in the provider profile of the processor that you use for real-time processing.
To review the naming convention of the file, see integration notes. To review the format of the file, see Capture File Format.
Charges file contains a list of charges associated with merchants. In the charges request file, a submitter sets a specific date when the charge becomes effective and will be picked up by the remittance process. Each charge in the file is going to be applied only once. Recurring charges should be re-submitted for every occurrence.
A subsequent response file informs the submitter whether the charges were accepted for processing or were rejected due to the validation issues. The charge is going to be picked up by the remittance process on or after the effective date (depending on when a process runs). The payments will be collected when an appropriate amount of funds due to the merchant is available (if money is not immediately available due to the negative balance of a merchant, it might take some time for a payment to actually be collected).
To submit charges to the system, you have to use a corresponding file. To review the naming convention of the file, see integration notes. To review the format of the file, see Charges File Format.
A subsequent response file informs the submitter whether the charges were accepted for processing or were rejected due to the validation issues. The charge is going to be picked up by the remittance process on or after the effective date (depending on when a process runs). The payments will be collected when an appropriate amount of funds due to the merchant is available (if money is not immediately available due to the negative balance of a merchant, it might take some time for a payment to actually be collected).
To submit charges to the system, you have to use a corresponding file. To review the naming convention of the file, see integration notes. To review the format of the file, see Charges File Format.
Charge payments file contains a list of payments that were collected from merchants on behalf of a reseller. Each record within the file includes a reference to a charge associated with a payment, as well as the amount that was withheld from a merchant. When an account of the merchant does not have sufficient funds, additional payments associated with a charge will be collected in the future and included in the subsequent charge payments files.
To review the naming convention of the file, see integration notes. To review the format of the file, see Charges Payments File Format.
To review the naming convention of the file, see integration notes. To review the format of the file, see Charges Payments File Format.
Tokenization Concept
See What is it? page for more information about Tokenization.
TidyPay provides a simple and convenient tokenization mechanism for merchants that deal with recurring payments and want to avoid storage of customer’s payment information (credit card and bank account numbers) within their own system. When a merchant uses any given credit card or bank account for the first time (as part of authorization, sale or credit operation), TidyPay will generate a unique identification number (token) associated with this payment information. If in the future, the merchant wants to issue another operation (e.g. another sale) on the same credit card or bank account, it is sufficient to pass to TidyPay just the value of the previously generated token and omit all of the account information fields.
TidyPay provides a simple and convenient tokenization mechanism for merchants that deal with recurring payments and want to avoid storage of customer’s payment information (credit card and bank account numbers) within their own system. When a merchant uses any given credit card or bank account for the first time (as part of authorization, sale or credit operation), TidyPay will generate a unique identification number (token) associated with this payment information. If in the future, the merchant wants to issue another operation (e.g. another sale) on the same credit card or bank account, it is sufficient to pass to TidyPay just the value of the previously generated token and omit all of the account information fields.
As well as simple token structure, extended structure enables integrators to easily derive information needed for reporting or system maintenance from the token itself.
For example, XVC01S0000000099990000014426594111001111.
For example, XVC01S0000000099990000014426594111001111.
| # | Name | Type | Description |
| 1 | Token Format | String(1) | Identifier of the extended format (always X). |
| 2 | Account Type | Enum(2) | Type of account. See possible values for more information. |
| 3 | Sequence Number | String(2) | For bank accounts only. Sequence number assosiated with the underline account number for cases when account number is not unique. For example, the same account number is used in different banks. |
| 4 | Tokenization Origin | Enum(1) | Identifies the system, underlines system, that generates the stock. See possible values for more information. |
| 5 | Bank Identifier | String(7) | Identifier of a bank that issue the credit card or that holds a bank account. |
| 6 | Reserved for Future Use | String(1) | |
| 7 | Account Number | String(16) | Tokenized account number. |
| 8 | First Four Digits | String(6) | The first four digits of credit card or bank account number (the last two digits of this value are reserved). |
| 9 | Last Four Digits | String(4) | The last four digits of credit card number or bank account number. |
A token generated by the gateway is comprised of 3 logical parts:
For example, a card number 4111111111111111 will be tokenized as VC84632147254611111111, where:
This structure enables integrators to easily derive information needed for reporting or system maintenance from the token itself.
- two-letter account type (see possible values for more information);
- tokenized account number;
- the last 4 digits of the actual account number.
For example, a card number 4111111111111111 will be tokenized as VC84632147254611111111, where:
- VC is the Visa account type;
- 8463214725461111 is a tokenized account number;
- 1111 are the last 4 digits of the actual card number.
This structure enables integrators to easily derive information needed for reporting or system maintenance from the token itself.
In this case token is associated with an internal account profile, which in addition to account number can contain other account information (for example, holder name, expiration date, routing number, address information). When transaction request is submitted, fields from the profile can be automatically used if they were not specified as part of the request. The disadvantage of the profiling approach is that profiles have to be updated if the customers’ information is changed, in particular the following fields:
- accountAccessory
- holderName
- isOrganization
- street
- city
- state
- zipCode
- countryCode
Message Formats
Usage:
| R | - | required in request/always present in response for direct debit transactions and credit card transactions of all levels (I, II, III). |
| O | - | optional in request/not always present in response. |
| C | - | conditional; conditions of the usage are defined below the corresponding section. |
| E | - | echo back from request; if present in request, it is present in response, if it is not present in request, it is not present in response. |
| R2 | - | required in request/always present in response for credit card transactions of level II and III only; optional for direct debit and level I credit card transactions. |
| R3 | - | required in request/always present in response for credit card transactions of III only; optional for direct debit and level I, II credit card transactions. |
| SR | - | required in request/always present in response for split transactions only. |
| I | - | for internal use only. |
| N | - | not used. |
| * | - | required fields in these specific sections are only required if this specific feature is used. |
Copyright ©
TidyPay. All Rights Reserved.
All Logos and Trademarks used or mentioned on this page are copyrighted property of their respective owners and are used for display purposes only.