Exit repetitive tasks: automate your imports!

Automating your imports with a CRON task

CRON task is a time-based job scheduler in Unix-like computer operating systems. Cron enables users to schedule jobs (commands or shell scripts) to run periodically at certain times or dates

It is indeed interesting to get the system to run recurent and repetitive tasks automatically instead of involving a manual intervention by the user.

Here is how you can make your life easier by automating your imports using a CRON task, allowing you to create and/or modify your products.

1. Configure your CSV import once to check that it works when ran manually
   A common configuration would be to identify products by their reference and to replace their properties

2. Download the attached file

3. Unzip the file and rename it cron_import_YYY.php (choose an recognisable name)

4. Edit the file to configure it

The first lines allow to modify the configuration:

// Cron task name
if (!isset($_GET['name'])) {
    $_GET['name'] = 'My task 1';
}

// CSV filename, all options must be set and tested in Store Commander
if (!isset($_GET['filename'])) {
    $_GET['filename'] = 'impDecli.csv';
}

// Do you need to delete the working file (TODO.csv)?
if (!isset($_GET['deletetodofile'])) {
    $_GET['deletetodofile'] = 0;
}

// Do you need to force the mapping? Optional. If used, enter the mapping name below.
if (!isset($_GET['forcemapping'])) {
    $_GET['forcemapping'] = '';
}

// Update only old products? This option will skip products recently updated
// (in minutes: set this option to 120 in order to update only products updated more than 2 hours ago)
if (!isset($_GET['olderthan'])) {
    $_GET['olderthan'] = 0;
}

// internal security key to use in CRON (ex: /modules/storecommander/XXXX/SC/cron_import_YYY.php?s_key=4hfsz65j4e68h4reh )
// Need to be changed !!
$localkey = '4hfsz65j4e68h4reh';

// the import result is sent to this email address
$email_address = 'info@yourwebsite.com';

// email wich send email (need to have same domain as website)
$email_from = 'contact@yourwebsite.com';

// email title
$email_title = '[CRON] Import SC';

// website url for the link inluded in the email.
$websiteURL = 'https://www.yourwebsite.com';

Then, add the file on your server in directory:
/modules/storecommander/XXX/SC/cron_import_YYY.php

You will then simply call this URL with the CRON task on your server:
/modules/storecommander/XXX/SC/cron_import_YYY.php?s_key=4hfsz65j4e68h4reh

/XXX/ corresponds to a serie of letters and digit corresponding to the security folder of you Sc installation.

Setting to fetch a CSV file on a distant URL for a CRON task

The CSV import file used to update your products via a CRON task is located on a distant URL.

Below is the data you will need to add to the CRON php configuration file to apply the process.

First of all, you need to set up the configuration file on your server. Refer to this article for the detailed procedure.

To then fetch the CSV file on a distant URL, add the following lines to the php configuration file, on the 4th line, just before // CSV filename:

$data = file_get_contents('http://Distant/File/URL.extension');
if($data == false) {
    die("Error: Unable to retrieve data. Check the url of your remote file and its accessibility.");
}
if(empty($data)) {
    die("Error: Empty data");
}
file_put_contents('../../import/your_csv_file.csv',$data);
unset($data);

The name of your local CSV file will need to be the one indicated in this setting:

$_GET['filename'] = 'file_to_import.csv'; 

🕰️Setting for the auto-import option in the cron task

To complete our current article on how to setup a CRON task to automate CSV imports, you can also configure multiple CRON tasks to import large CSV files multiple times.

To do so:

• In the CRON php file, configure the option 'olderthan' to 120 minutes for example. This way files that have been modified less than 120 minutes ago will not be processed.

• The option "deletetodofile' should be set to '0' allowing the CRON tasks to continue the same import instead of starting from the beginning of your original file.

• Within the Import panel in Store Commander (top panel), modify the number of lines to import by a quarter of the total lines existing in your CSV file. For instance, if the file contains 1956 lines, configure the option 'number of lines' to 500 for 4 CRON tasks (500*4=2000 lines to be processed in total).

• Then run the CRON tasks multiple times in a row (in our example: 4 times every 20 minutes. You need to keep a 20 minutes interval to avoid importing the same data multiple times).

This way, your large CSV files will be processed in 4 parts rather than one. You can of course modify the above settings to suit your needs, depending on the size of the CSV files.