Batching and routing in ShareScan

Batching and routing is the ShareScan capability to split a scan job into separate batches and handle/process/route the output documents independently of one another.

For instance, you can scan a stack of sheets and have a separate document created out of every ten-page batch. If you scan, say 40 pages, you can have 4 output documents created out of this single scan job. For every document, you can specify file format, connector target, and so on.

All ShareScan connectors support batching.

All connectors support batching in a scenario when either:

There is no capture user interface shown on the MFP screen (That is, MFP end-users do not need to set file format, document metadata: they simply press the button of a connector and can leave the device as soon as all sheets have been scanned), or
When capture user interface is shown on the MFP screen, but Background processing is enabled. Background processing means that MFP end-users do not have to wait at the device until the output documents are created. Processing is taken over by the ShareScan Manager.

There are two basic types of batching:

Connector-level batching (also known as 'native' batching): when document (meta)data (file naming, format, author, subject, and so on, depending on the actual connector) MFP screens are shown only once, and user choices are applied to all output documents for that scan job. (Using the example above: you choose PDF file format once, when your first 10-page batch is scanned, and all 4 documents created out of your scan job will be PDFs).
Global batching (also known as 'automatic/generic' batching): lets you set document data for each output document in the scan job. You can select PDF file format for your first 2 documents, then pick DOC for your third one, and PDF again for your last 10 pages.

What is Batching?

Batching or batch processing involves the automatic execution of scanning jobs on MFP devices with the help of pre-written scripts/algorithms without the need for human or any other form of intervention. If you have a large stack of hard copy documents to scan, you can easily split them into several smaller batches according to your different scanning needs. You can direct these diverse scanning tasks to multiple devices/destinations and they will be completed without interruption.

Major version batching differences

There is a difference in the way ShareScan handles batching scenarios in pre-5.2 versions and in version 5.2 and higher.

Pre-5.2 version batching

Versions prior to 5.2 implemented so-called 'native' batching: all pages/documents and batching data are sent to the connector in one pass, and the connector is responsible to process every document selecting the proper batching data. The procedure is the same in both Online and Background mode.

Not all connectors have native batching capabilities - and some are capable of this when in Express mode (no end-user interaction is required at the MFP).

Version 5.2 batching

To make batching available for connectors without this capability and to eliminate the need to modify existing third-party connectors, ShareScan 5.2 offers 'automatic' batching.

This model is driven by a Document Service assigned to the connector profile: every document created from the scanned pages is sent sequentially to the connector with the proper routing and indexing data belonging to the document. The connector behaves exactly the same way as it was used with a simple scanning job with the pages of the first document, then the second, and so on. This also means that if configured so (non-Express mode), all connector forms are displayed on the device. But generic forms or those belonging to the document services are not displayed again before sending the second, third, and so on. document to the connector.

Connector functionalities

Original connector functionalities are redesigned since ShareScan 5.4 to make their usage more comfortable.

Connectors can be grouped based on how their functionalities are changed:

CORRECTED original functionality:

Connector

SupportsNativeBatching

InExpressMode

SupportsNativeBatching

InUIMode

ExchangeMail

ExchangeFax

SMTP via LDAP

SMTP Fax

Worksite

other Kofax connectors

Yes

EXTENDED original functionality:

Connector

SupportsNativeBatching

InExpressMode

SupportsNativeBatching

InUIMode

QuickConnect

ScanToFile

SharePoint

Yes

Native vs automatic batching

To help you choose which batching behavior to use, consider the following:

Not all connectors have native batching capabilities. Native batching is only available for a limited set of connectors (for some, in Express mode only) - for details, see the summary table below. Native batching support (in Express mode) is introduced by ShareScan 5.2 for the following connectors: Exchange Mail, Exchange Fax, SMTP via LDAP, SMTP Fax and Worksite.
Less time spent at the device. In case of native batching, any routing, storing, or authentication-related tasks are solely based on the values arriving from Data Publishing. Thus, once properly configured, there is less need for user interaction, so the overall time spent at the MFP side decreases.
Automatic batching covers more connectors, even the ones incapable of native batching.
Automatic batching gives you more MFP-side interaction control over your batches. This practically means that end-users interact with routing and indexing-related MFP screens for each document in the batch.
Automatic batching is a better choice if the output documents in your batch are not more than a couple of pages and you have the proper infrastructure to facilitate quick processing.

To make the choice between the two batching modes available, you need to access Shared Manager Settings (under Advanced ShareScan settings) and add the relevant connector id to either ConnectorsAllowedToSelectBatchBehaviorInOfflineMode or to ConnectorsAllowedToSelectBatchBehaviorInOnlineMode depending on how you want to handle Background processing (see below). Connectors added to these lists display an Automatic batching check box in the Services group on the Settings pane if batch processing is configured (that is, the connector profile has a document service assigned to it, and this service is configured to split the scanned pages into more documents).

Native batch processing support depends on two connector capabilities:

Name	Description
SupportsNativeBatchingInExpressMode	The connector supports native batch processing only when the connector profile is configured as an express profile (when the connector profile displays no screens on the MFP, so there is no user interaction).
SupportsNativeBatchingInUIMode	The connector supports native batch processing when the connector profile is configured to display connector screens on the MFP (user interaction is required at the device).

These capabilities are described as service settings for every connector in the <connector name>Services.xml files located under the connector data folder.

In this file the supports-NativeBatching setting specifies the SupportsNativeBatchingInExpressMode capability and the supports-NativeBatchingInUIMode setting specifies the SupportsNativeBatchingInUIMode capability.

For example, Worksite connector - supports native batching in Express mode only:

file name: WorksiteConnectorServices.xml

file location %programdata%\Kofax\ShareScan\Connectors\WorksiteConnector\Data

Settings:

supports-NativeBatching="true"
supports-NativeBatchingInUIMode="false"

If these settings are missing from this file, then ShareScan behaves as if the connector did not support native batch processing at all. (This is the case with third-party connectors).

If the connector profile has no Data Published value coming from the scanned document, ShareScan selects native batch behavior. ShareScan does not display an error message on whether the connector supports native batching or not. As this is not a document dependent batch situation, the Data Published value is the same for every document in the batch. This can occur when a document service Published key is not mapped to a connector Published key in the connector profile (for example, if the connector uses only Client and Matter Published keys coming from Cost Recovery system).

Do not modify these factory connector settings.

These settings are described in this file so that third-party connector batch processing behavior could be modified simply on connector level.

You can also modify connector batch processing behavior by configuring new advanced settings related to batch processing behavior (see later).

Automatic batching and background processing

If you are using automatic batching, connector behavior depends on the Background processing setting.

If Background processing is enabled the connector behaves as if the user had clicked the Done button on the last screen and had scanned the pages of the next document with the same settings.

If Background processing is disabled the connector behaves as if the user had clicked the New document button on the last screen and had scanned the pages of the next document with the same settings.

Selection of batch processing behavior

ShareScan is able to determine if batching is needed: if a document service (assigned to the given connector profile) uses document splitting, the ShareScan system treats this as such a case.

ShareScan automatically selects either native or automatic batch behavior. Unsupported configurations display an error message.

Based on connector batch supporting capabilities, configuration and other settings, it automatically selects the batch processing behavior to be applied.

Connector	Express mode (no end-user interaction)		Non-express mode (end-user interaction)
Connector	Online	Background	Online	Background
Quick Connect, SharePoint	Native	Native	Native	Native
Exchange Mail, Exchange Fax, SMTP via LDAP, SMTP Fax, Worksite	Native	Native	N/A	Automatic
Other Kofax connectors	N/A	Automatic	N/A	Automatic
Third-party connectors	Native if implemented, otherwise N/A*	Automatic* or Native and selectable by an advanced setting	N/A (possible to override by advanced setting*)	Automatic* or Native, selectable by an advanced setting

*If there are no connector DP keys mapped to the DP keys of a document service assigned to the connector profile, the Manager executes native batching to preserve compatibility.

To use batching with connectors, you do not need to perform any specific actions during connector profile Data Publishing mapping configuration. The Data Publishing can be configured in the regular way, and when the batching is available from the Document Service side, and the specific connector allows for batch processing, the connector processes data in batched manner.

Be aware that when batch processing is executed, exceptions do not stop processing, but they are traced, and an error report is included into the notification message of the Notification service, if Notification is configured. For more information on the Notification service, see Notification service.

The below sections outline the connector-specific settings required for the batch-based routing and indexing to work.

Connector capabilities related to batch processing

The following table describes the capabilities Kofax connectors have and how ShareScan handles third-party connectors regarding these capabilities:

Connector	SupportsNativeBatching InExpressMode	SupportsNativeBatching InUIMode
QuickConnect ScanToFile SharePoint	Yes	Yes
ExchangeMail ExchangeFax SMTP via LDAP SMTP Fax Worksite	Yes	No
Other Kofax connectors	No	No
3rd party connectors	No	No

Batch processing scenarios

The following tables summarize automatic batch processing behavior selections based on connector capabilities and default capability settings.

Background processing DISABLED

Connector capabilities:

Connector capabilities

Express mode

(no UI)

Normal mode (with UI)

SupportsNativeBatchingInExpressMode

and

SupportsNativeBatchingInUIMode

Native

SupportsNativeBatchingInExpressMode

and

NOT SupportsNativeBatchingInUIMode

Native

Not supported

(*)

NOT SupportsNativeBatchingInExpressMode

and

NOT SupportsNativeBatchingInUIMode

Not supported

(*)

Not supported

(*)

Default capability settings:

Connector

Express mode

(no UI)

Normal mode (with UI)

QuickConnect / ScanToFile

Kofax SharePoint

Native

Exchange Mail and Fax

SMTP via LDAP and Fax

Worksite

Native

Not supported

(*)

Other Kofax connectors

3rd party connectors

Not supported

(*)

Not supported

(*)

Background processing ENABLED

Connector capabilities:

Connector capabilities

Express mode

(no UI)

Normal mode (with UI)

SupportsNativeBatchingInExpressMode

and

SupportsNativeBatchingInUIMode

Native

SupportsNativeBatchingInExpressMode

and

NOT SupportsNativeBatchingInUIMode

Native

Automatic

(*)

NOT SupportsNativeBatchingInExpressMode

and

NOT SupportsNativeBatchingInUIMode

Automatic

(*)

Automatic

(*)

Default capability settings:

Connector

Express mode

(no UI)

Normal mode (with UI)

QuickConnect / ScanToFile

Kofax SharePoint

Native

Exchange Mail and Fax

SMTP via LDAP and Fax

Worksite

Native

Automatic

(*)

Other Kofax connectors

3rd party connectors

Automatic

(*)

Automatic

(*)

Advanced settings for batch processing

There are two advanced settings which enable ShareScan Administrators to select the batch processing behavior (automatic or native batching) for a connector profile. Use these settings to support batch scenarios by default unsupported by ShareScan (mostly third-party connectors).

ConnectorsAllowedToSelectBatchBehaviorInBackgroundMode
The value of this setting is a list of connector IDs where batch behavior can be selected if Background processing is enabled.
ConnectorsAllowedToSelectBatchBehaviorInOnlineMode
The value of this setting is a list of connector IDs where batch behavior can be selected if Background processing is disabled.

If a connector ID is in the list, then you can select batch behavior at the profiles of this connector.

Connector IDs are stored in the registry under the following key:

HKEY_CLASSES_ROOT\eCopyShareScan\Connectors\<ConnectorID>.MainControl\ConnectorID

If the connector ID appears in one of these settings' value, then an Automatic batching check box is displayed under the Services group on Settings pane in the Administration Console.

This check box is displayed only if a batch processing is configured (a document service is assigned to the connector profile configured to split the scanned pages into more documents).

If this Automatic batching check box is selected, then Automatic batching is used, otherwise Native batching is used. If the ConnectorsAllowedToSelectBatchBehaviorInBackgroundMode value contains the connector ID, then the check box is displayed if Background processing is enabled at the profile. If the ConnectorsAllowedToSelectBatchBehaviorInOnlineMode value contains the connector ID, then the check box is displayed only if Background processing is disabled at the profile.

PublishingKeysForAllBatches Registry Setting

If a Data Publishing value is set before scanned pages are split into more documents, it is applied only to the first document.

The rest of the documents will have the default value of the Data Publishing key.

This is the case when a Lookup Extender profile is processed before the document service/extender profile configured with split option or with connector-based splitting (that is, splitting by Number of Pages, File Size, Blank Page).

The new PublishingKeysForAllBatches registry setting makes it possible for these types of Data Publishing values to be applied to all batch documents.

To achieve this, the names of these Data Publishing keys must be specified (separated by semicolon) as a value of this registry setting.

Key = HKLM\Software\{Wow6432Node\}Kofax\ShareScan\ShareScanManager\

Name = PublishingKeysForAllBatches

Data = Fully qualified source DP key names separated by semicolon

For example:

LookupDocumentService.default.GEO

where LookupDocumentService is the name of the extender, default is the name of the profile and GEO is the Data Publishing key name for that profile.

For all Data Publishing source keys listed in this setting, the connector uses the Data Publishing source key value of the first document for all documents when they are mapped to a connector Data Publishing key.

Connector Configurations and Example Scenarios

Exchange Mail

The connector uses batched published data when the Default recipients source is configured for Data Publishing or Default recipients and Data Publishing on the Sending Options tab of the connector. For more information on the connector configuration, see eCopy Connector for Microsoft Exchange.

When using batching with Exchange Mail or Exchange Fax connectors, selecting the correct Display option is important. Kofax recommends selecting Express to Default recipients option when batching is enabled. Using this prevents the Send screen from displaying and the workflow speeds up. However, in this case, changes to the recipients, subject or note fields at runtime is not possible, as the fields are populated from data publishing and from the default settings.

If the subject of the sent email contains the $$SUBJECT$$ text, this substring is replaced everywhere with the actual value of ExchangeMail_Subject for each batch. Similarly, when the notes field contains the $$NOTE$$ text, this substring is replaced everywhere with the actual value of ExchangeMail_Note key for each batch. These special tokens can be used manually as well when the Send screen appears.

If you select the Display form option from the Display options combobox, the Send screen is always displayed. This setting can be useful when the user needs to change the subject or note fields at runtime. The To/Cc fields on the Send screen will not contain the recipients coming from data publishing keys (ExchangeMail_To, ExchangeMail_CC). This is because these keys can have different values for each batch. However, if the user types in an additional recipient it will be added to the recipient list for each batch.

The batching support of this connector is limited for the following published keys:

ExchangeMail_To
ExchangeMail_Cc
ExchangeMail_Subject
ExchangeMail_Note

Fax via Exchange

The batching support of this connector is limited for the following published keys:

ExchangeFax_To
ExchangeFax_Subject
ExchangeFax_Note

SMTP via LDAP

The batching support of this connector is limited for four published keys:

SMTPMail_To
SMTPMail_Cc
SMTPMail_Bcc
SMTPMail_Subject

Fax via SMTP

The batching support of this connector is limited for a single published key:

SMTPFax_To

Worksite

The Worksite connector can support batching for those profile configurations which do not require interaction from the end-user. Thus, you have to have a single destination set, or you must have folder navigation disabled (meaning no navigation screens are displayed on the MFP screen). For properly routing the processed documents, you must also provide the relevant credentials, so Worksite can perform the tasks without having to prompt for user interaction.

To achieve a fully automated batching and routing in your workflow, the following configuration settings are required:

Ensure that the Worksite connector profile contains only a single destination.
Select the single destination, and click Edit.
On the Navigation tab of the Edit Destinations pane, set Auto-select or Specify mode.
On the Navigation tab of the Edit Destinations pane, ensure that the Provide a storage confirmation screen option is cleared.
On the Navigation tab of the Edit Destinations pane, ensure that the Provide autonavigation property screen as needed option is cleared.
On the Attributes tab of the Edit Destinations pane, the document profile attribute population method must be set to Auto-index.

When all the above requirements are met, the Worksite connector supports batching for all of the published keys.

For more information on Worksite connector customization, see eCopy Connector for iManage WorkSite.

SharePoint

The following configuration settings are required for batching support:

Set the Show option of the Columns tab to Auto-Index.
The Retrieve Content Type from Data Publishing option is configured on the Columns tab
You must set at least one column as Published (a highlighted key must be displayed before the column name).

If there is no column configured as a Published column, the Column Information form is displayed independently from the batching mode and the column values specified by the user are set to all documents in the batch.

Auto-select location is configured

In this case the location where a document has to be stored can come from data publishing.

Documents in the batch can be stored to different locations depending on the batched published value without any user interaction. The Navigation form is not displayed.

If the ColumnInformation is not displayed in this case, the document is stored with content type and column values specified on SharePoint server within Administration Console. Otherwise they are retrieved from data publishing depending on Retrieve Content Type from Data Publishing setting, Auto-index show option and column settings.

Retrieve Content Type from Data Publishing

In this case the content type of the document to be stored can come from data publishing. Documents in the batch can be stored with different content type (and so different column values) depending on the batched published value without any user interaction. The ColumnInformation form is not displayed so the user has no possibility to set any column values on the device even if All, Required or Auto-Index show option is configured.

Auto-Index show option is configured with at least one column as Published column.

In this case, the connector attempts to use batched published data in those columns that are configured as Published columns (signaled by the highlighted key icon before the column in the column list) in the automatically selected runtime content type.

The runtime content type is

the content type retrieved from data publishing, if Retrieved Content Type from Data Publishing setting is also configured and the content type retrieved from Data Publishing exists at the runtime location
the default content type of the runtime location, if Retrieved Content Type from Data Publishing setting is also configured and the content type retrieved from Data Publishing does not exist at the runtime location
the default content type of the runtime location, if Retrieved Content Type from Data Publishing setting is not configured

Runtime location is

the location retrieved from data publishing,
the location selected by the user on Navigation form, or
the location specified as Hyperlink in Admin Console if Navigation was not enabled at the profile.

In this case the ColumnInformation form is not displayed and the connector tries to store the document with the proper batched published column values.

The ColumnInformation form is displayed in batching mode only if

there is no column configured as Published column,
Retrieve Content Type from Data Publishing is not configured and
Auto-select location is not configured.

In this case the column values specified by the user on ColumnInformation form will be set to all documents in the batch as in the earlier versions of the connector.