XSM     Current stable release: 6.94   Beta release: n/a

  1. Select your platform and download
  2. Take a look at the Quick Install Guide
  3. Register on XSM Extranet and generate your activation key online

NOTA : These versions are not restricted demo versions but full-featured versions. An activation key is needed to run XSM, otherwise XSM will prompt "Press ENTER" on console. For a quick evaluation, you can freely download and test XSM without activation keys in interactive mode (no batch) ; in this case, it is not necessary to register.

Check XSM ChangeLog for last features and bug fixes.

Plateform
Operating System CPU Architecture Product Version Evolution &
Support
Windows
Windows 2012, 2008, 2003, 10, 8, Seven, Vista, XP x86 XSM v6.94
Linux         :: Linux users, please choose the appropriate glibc level ::
RHEL8, Centos8, FC28 (kernel ≥ 4.18.0 glibc ≥ 2.18) x86 XSM v6.94
RHEL7, Centos7, FC20 (kernel ≥ 3.10.0 glibc ≥ 2.17) x86 XSM v6.94
RHEL6, Centos6, FC16 (kernel ≥ 2.6.32 glibc ≥ 2.12) x86 XSM v6.94
RHEL5, Centos5, FC9 (kernel ≥ 2.6.18 glibc ≥ 2.5) x86 XSM v6.94
RHEL4, Centos4, FC3 (kernel ≥ 2.6.9 glibc ≥ 2.3) x86 XSM v6.77
SuSE SLES 12 (kernel ≥ 3.12.28 glibc ≥ 2.19) x86 XSM v6.94
SuSE 11 z/Linux (kernel 3.0.76 glibc 2.11.3) IBM z/S390x XSM v6.93
SuSE9 (kernel ≥ 2.6.18 glibc ≥ 2.3) IBM PPC XSM v6.68
AIX
AIX 6.1 (compatible AIX 7) P/systems XSM v6.94
AIX 5.3 P/Series XSM v6.94
Sun Solaris
SunOS Solaris 10 SPARC XSM v6.94
SunOS Solaris 8 (9 compatible) SPARC XSM v6.94
SunOS OpenSolaris 11 x86 XSM v6.94
Misc.,   HP-UX   OS/400   z/OS
HP HP-UX 11i v3 Itanium ia64 XSM v6.68
OS/400 AS400 IFS i/Series please contact us for trial
z/OS 1.9 Unix System Services (USS) z/Series please contact us for trial
Development API for Windows (DLL) and UNIX/Linux (lib.so) please contact us for trial
Can't find your favorite platform on this page ?
Just ask and We'll compile !

CHANGE LOG


----------------------------------------------------------------------------

   Release Notes for Version 6.94 RC2 :
   02-May-2023 

Bugs fixed:  (Linux x86)
    When excuted by crontab or scheduler, XSM was complaining about wrong activation key.  Fixed. 

----------------------------------------------------------------------------

   Release Notes for Version 6.94 :

   02-Jan-2022 

	Compiled on RHEL8/CENTOS8
   
   03-Jul-2020

New features:
	--outfiledup=             Writes duplicate records to a file, according to OMIT DUPLICATE KEYS/RECORDS option.
       or
    OUTFILDUP    outfile                                           
       or
    OUTFILDUP    DD:variable

    --outfiledup-record1      To use with --outfiledup= : Also writes first record that will compare to duplicates.  
       or                     In option is not specified, then only next duplicate records are written to duplicate file.
    --dup-rec1
       or
    OPTION  WRITEFIRSTDUPLICATE
        
  Misc:
	Improved control on sort statements and options.  
    
    
----------------------------------------------------------------------------

   Release Notes for Version 6.93 RC3 :
   18-Feb-2020

Bugs fixed:
	Work buffer too small for fullpathnames (path + filename) greater than 110 chars. Increased to 260 char.

----------------------------------------------------------------------------

   Release Notes for Version 6.93 :
   06-June-2017

Misc:
	Improved control on sort statements and options.
	Hardening control on FIELDS positions and length.
	Improved warning/error messages

Bugs fixed:
	OMIT COND=(a,OR,b,AND,c,...) buffer was too short for big (+20 lines) conditions. Increased to 2000 char.
	
	Bug on parenthesis in conditions COND=(a,AND,(b,OR,c)). Fixed
	
	Bug on multiple INCLUDE/EXCLUDE statements. Fixed
	
	Bug on --sort command line option. fixed.
	
	Improper message "XSM004W Warning: No SORT/COPY/MERGE statement specified, defaulting to SORT FIELDS=ALL"
	when using no parameter file. Fixed.

New features:
	OUTREC FIELDS=(...) fixed and improved. Now works with VFIELDS.
    
	--veryverbose gives more and better details.
	
	Now tolerates SORT FIELDS=COPY for IBM SORT compatibility, equivalent to OPTION COPY.
	
	Now tolerates default numeric format COND=(1,3,GT,123)
	           as equivalent to          COND=(1,3,GT,N'123'). for IBM SORT compat.
	
	Warning if multiple OUTFIL DD use same variable or filename.

----------------------------------------------------------------------------

   Release Notes for Version 6.92 :
   13-June-2015

New features:
         INCLUDE/EXCLUDE now work now with VFIELDS (CSV files)
         --veryverbose show more details and counters 

----------------------------------------------------------------------------

   Release Notes for Version 6.91 :
   25-May-2015

Misc:
         Performance improvement again and again ...
           hxsm690_64     sz=100GB  Elapse: 6664 sec.
           hxsm691_64     sz=100GB  Elapse: 3721 sec ... nearly 2 times faster! 
           (1 x Intel Xeon E5-1620, 64 GB ram,  RH7)

Bugs fixed:
         Bad buffer size over 500GB input files. All systems

New features:
         --skip-headers=n : skip n records from all input files
         (when --skip-header=n skip n records only from first input file.

----------------------------------------------------------------------------

   Release Notes for Version 6.90 RC5 :
   13-May-2014


Bugs fixed:
         Segmentation fault fix : Thread stack size tuned. AIX 6.1
         Wrong count on OMIT DUPLICATE RECORD with VFIELDS. Fixed

New features:
         Ported to Windows 64-bits. Here it is!

----------------------------------------------------------------------------

   Release Notes for Version 6.90 RC4 :
   26-April-2014


Bugs fixed:
         Segmentation fault fix : Thread stack size tuned. AIX 6.1

New features:
         Optimization on OMIT DUPLICATE RECORDS + FIELDS=ALL operation

----------------------------------------------------------------------------

   Release Notes for Version 6.90 RC3 :
   15-March-2014


Bugs fixed:
         Segmentation fault fix : Thread stack size increased. AIX 6.1


----------------------------------------------------------------------------

   Release Notes for Version 6.90 RC2 :
   13-March-2014


Bugs fixed:
         Memory allocation improvement. All systems

New features:
         Display Error message then stop when RECFM=V and record length is smaller
         than sort keys, to prevent segmentation fault.


----------------------------------------------------------------------------

   Release Notes for Version 6.82 RC4 :
   26-Oct-2013


Bugs fixed:
         Limit threads and memory allocation on big machines with many CPUs.  All systems


----------------------------------------------------------------------------

   Release Notes for Version 6.82 RC2 :
   08-Oct-2013


Bugs fixed:
         Wrong Serial/Number displayed on AIX6.1. Fixed


----------------------------------------------------------------------------

   Release Notes for Version 6.82 :
   28-Sep-2013


Bugs fixed:
         1) Memory allocation improvement. All systems

	 2) Include/Exclude filtering bug. Fixed
		

----------------------------------------------------------------------------

   Release Notes for Version 6.81 :
   09-Jun-2013


Bugs fixed:
         1) AIX, Solaris, Linux : hang/segmentation fault could occur on low memory condition.  Fixed.

         2) input/output file mismatch with COPY operation. Fixed.
		

----------------------------------------------------------------------------

   Release Notes for Version 6.80 :
   03-Jun-2012


Bugs fixed:
         1) Error in parsing AND/OR conditions in INCLUDE/EXCLUDE statements. Fixed.

         2) Could not use COMP-3 fields in the include or omit. Fixed.
		
New features:
        Up to version 6.79, XSM Serial Number used for licence activation was calculated on IBM Hardware Serial
        Number and LPAR Id. Thus it was not possible to move a LPAR running XSM from a system to another.
        Upon clients requests and specialy for clients using IBM POWER Live Partition Mobility (LPM), XSM from
        v6.80 is now only bound to AIX Operating System and no longer to Hardware, making it possible to move 
        a LPAR during XSM jobs.


----------------------------------------------------------------------------

   Release Notes for Version 6.79 :
   18-Apr-2012


Bugs fixed:
         1) SUN SPARC Solaris 5.8/5.10, IBM Power AIX 5.3, 6.1 : Wrong Memory allocation.  Fixed.

         2) SORTOFn for selective output (all systems). Fixed.
		
Misc:
         Performance improvement again and again ...
           /usr/bin/sort  sz=1GB  Elapse: 314 sec.
           hxsm671_64     sz=1GB  Elapse: 113 sec.
           hxsm679_64     sz=1GB  Elapse:  77 sec.
           (IBM p510 lcpu=4 mem=7168MB ent=0.30 AIX6.1)

----------------------------------------------------------------------------

   Release Notes for Version 6.78 :
   05-Dec-2011


Bugs fixed:
         1) SUN SPARC Solaris 5.8/5.10 : XSM007E_0174 Cannot open Input file.  Fixed.
		
         2) IBM Power AIX 5.3, 6.1 : Wrong Memory allocation when low available free memory.  Fixed.
		

----------------------------------------------------------------------------

   Release Notes for Version 6.77 :
   01-Apr-2011


Bugs fixed:
         1) OUTREC FIELDS format incorrectly parsed.

         2) inputfile read error could occur on +30GB (SPARC Solaris)

Misc:
         Performance improvement on Multitasking and I/O operations 
         Ex: sort 10 GB (100 millions records x 100 bytes, no sortwork), Intel Core2 E8500, 2GB Ram
         Linux-64:
            V671 :   total 370 sec.
            V677 :   total 300 sec.
         W32:
            V671 :   total 850 sec.
            V677 :   total 550 s. 

New features:
        API released.   Included as part of XSM Software licence.
        Delivered as static and dynamic libs + C/C++ header.
        Compatible with all programing languages which are standard Link Edit compliant
        (C, C++, VB, .Net, Pascal,  ...)



----------------------------------------------------------------------------

   Release Notes for Version 6.71 :
   18-Feb-2010


Bugs fixed:
         1) Text records too long and IOERROR IGNORE could cause a crash in sort operation.

         2) Too many long file names in INPFIL statements caused a crash.
            Buffer for input filenames was 4K, too short for +100 long pathnames. 
	    Now use dynamic allocation : no more limit on number of input files.

         3) VFIELDS (all systems) : on large files, following error could randomly occur:
            "XSM063E The file xxxxx/srtw3254.03a is not sorted at record # yyy"

         4) minor correction in INCLUDE/EXCLUDE operation precedence
   

New features added:
      --skip-head=n     :  will drop the first n records of the first input file
                           Usefull to throw away a header/title

Minor updates :
          1) Linux : no more need to be root to install hxsm 

          2) Any unknown parameter is flagged as an error [instead of ignored],
             and the job is cancelled

          3) if RECFM=V, the record length is automatically increased by 2,
             thus permitting the user to give a LRECL without dealing with
             'carriage-return' and 'line-feed' chars.

          4) Memory, sortworks, threads optimization for huge files (+100GB) 


----------------------------------------------------------------------------

   Release Notes for Version 6.68 :
   01-Apr-2009

   Port to HP-UX 11i v3 IA64 Itanium (uname says: HP-UX B.11.31 U ia64 HP-UX)

----------------------------------------------------------------------------

   Release Notes for Version 6.68 :
   15-Mar-2009

Bug fixed:
      - VFIELDS (all systems)

Major performance improvement. We now sort 1 Gigabyte in less than a minute on an inexpensive PC!
      v6.62 : XSM031I END OF JOB 1gt.xsm                           10000000 rec.:   80 SEC.
      v6.68 : XSM031I END OF JOB 1gt.xsm                           10000000 rec.:   42 SEC.
      (Intel Core2 2.2Ghz, 2GB RAM, 1 x IDE 7500rpm, Solaris 10 x86)


----------------------------------------------------------------------------

   Release Notes for Version 6.64 :
   19-Feb-2009

Bug fixed:
      - Free storage detection (AIX, Solaris)
      - Thread stack size allocation (AIX)
      - dynamic STORAGE allocation (all systems), affecting sortworks size + quantity

New features added:
      Verbose mode (-vv) now shows real memory + working dirs + threads being used.

----------------------------------------------------------------------------


   Release Notes for Version 6.62 :
   07-Feb-2009


Bug fixed:

    - OUTREC option
   With this option, one can reformat the final output records with padding and constant fields, 
   and with selected fields from the sorted records.

   Option may be speciped in 3 ways :

   format #1 :  command line option : --outrec=(specs[,specs...])
   format #2 :  Parameter file : OPTION OUTREC=(specs[,specs...])
   format #3 :  Parameter file : OUTREC FIELDS=(specs[,specs...])[,RECFM=...][,LRECL=...]

   where  specs ::= start,length[,type,offset]
           start  = field start in input record      (mandatory, from 1)
           length = field length                     (mandatory, may be < 0 for last field if RECFM T)
           type   = field type                       (optionnal)
           offset = field offset in output record   (optionnal, from 1)

   If  is omitted in 1st spec, they must be omitted in all specs,
       and fields are concatened in output record


   Example 

       hxsm ... '--outrec=(14,20,1,,..6)'
       OPTION OUTREC=(14,20,1,6,...)
       OUTREC FIELDS=(14,20,1,6,...)

       are equivalent to :

       hxsm ... '--outrec=(14,20,C,1,1,6,C,21,...)'
       OPTION OUTREC=(14,20,C,1,1,6,C,21,...)
       OUTREC FIELDS=(14,20,C,1,1,6,C,21,...)
                                            
   ---------------------------------------------------------------------------

   Padding and constant fields:

   The output record may include fields that are not in the corresponding input record.
   These fields may be :
          alpha-numeric fields, padded with a letter, a digit, a space, or  any other byte
          zoned-numeric fields, padded with a digit (most of time 0)
          packed-numeric fields, padded with a digit (most of time 0)

   In this case, 'specs' syntax are as follows:

     1)   any_char,length,S,offset   : alpha field  padded with spaces
     example:
          OPTION OUTREC=(...,X,10,S,42,...)    # cols 42 to 51 padded with spaces
          '--outrec=(...,X,10,S,42,...)'       # same

     2)   letter,length,C,offset   : alpha field  padded with 'letter'
     example:
          OPTION OUTREC=(...,Z,10,C,42,...)    # cols 42 to 51 padded with letter 'Z'

     3)   digit,length,N,offset   : numeric field padded with 'digit'
     example:
          OPTION OUTREC=(...,0,10,N,42,...)    # cols 42 to 51 padded with '0'

     4)   value,length,B,offset   : alphanumeric field padded with byte decimal 'value' (0..255)
     example:
          OPTION OUTREC=(...,0,8,B,42,...)    # cols 42 to 49 padded with binary zero
          OPTION OUTREC=(...,20,8,B,42,...)    # cols 42 to 49 padded with (ascii) space
          OPTION OUTREC=(...,255,8,B,42,...)    # cols 42 to 49 padded with HIGH-VALUE (255)

     5)   digit,length,P,offset   : packed numeric field value 'digit'
     example:
          OPTION OUTREC=(...,0,3,P,42,...)    # cols 42 to 44 = x'00000C' (packed 0)
          OPTION OUTREC=(...,-1,3,P,42,...)   # cols 42 to 44 = x'00001D' (packed -1)

     6)   digit,length,Z,offset   : zoned numeric field value 'digit'
     example:
          OPTION OUTREC=(...,0,3,Z,42,...)    # cols 42 to 44 = x'F0F0C0' (zoned ebcdic +0)
                                                                x'30307B' (zoned ascii +0)
          OPTION OUTREC=(...,9,3,Z,42,...)    # cols 42 to 44 = x'F0F0C9' (zoned ebcdic +9)
                                                                x'303043' (zoned ascii +9)
          OPTION OUTREC=(...,-1,3,Z,42,...)   # cols 42 to 44 = x'F0F0D1' (zoned ebcdic -1)
                                                              = x'303071' (zoned ascii -1)

----------------------------------------------------------------------------


                hxsm V658 - Multitasking - Nov 2007 -  


   Supported platforms:

        IBM pSeries RS6000 Power3-Power5  - AIX 4.3 AIX 5.2    : 32 bits
        IBM pSeries RS6000 Power3-Power5  - AIX 5.3            : 32/64 bits
        IBM Z-series  - zLinux-390 (Suse 9)      : 32 bits     kernel=2.6.5
        IBM Z-series  - zLinux-390 (Suse 9)      : 64 bits     kernel=2.6.5
        Intel Pentium - Linux 2.6 (>FC4)         : 32 bits     kernel>=2.6.11  glibc>=2.3.5
        Intel Pentium - Win32 x86 (NT4, W2K, XP) : 32 bits
        Sun SPARC Solaris 8, solaris 9, solaris 10 : 32/64 bits

   Release Notes for Version 6.58 :
   26-Oct-2007

   Several bugs fixed:

      - sortwork allocation (Solaris 8)
      - dynamic STORAGE allocation (all systems)
      - KEEP_ORDER (all Systems)
      - improved traces for debug mode (All Systems)

----------------------------------------------------------------------------

   Release Notes for Version 6.56 :
   Nov-2006

This version is a major enhancement of XSM
'Basic' version 4 limited to 4GB input file size is abandoned.
From version 6, XSM is now released in 'Pro' version without software limitation.


A)  New features added :
========================

  1 - Almost all options can be used in a command line, using GNU style long options 
      (i.e.           --lrecl=250    instead of   -l 250  )

  2 - New field types :

          numeric ('N')   : fixed length numeric fields with a leading floating sign
                            (-ie  +1   -124  ....)
          zoned decimal ('Z' or 'ZD')  : fixed length numeric fields with a trailing sign
          on the last digit ( "PIC S99999 USAGE IS DISPLAY"  ) ...
          Both methods (Microfocus-like and ACU COBOL-like) are supported.

    Note: This is different from 'packed decimal' ('P' or 'PD', COBOL "COMP-3")

  3 - Support of MicroFocus COBOL special variable format (known as "MFCOBOL" or "MFVariable") : RECFM=M

  4 - Support of multiple format for all input files and the output file :
      each input file may have its own record format (F,T,M) and its own record length
      the output file may have its own record format (F,T,M) and its own record length

  5 - Aditionnal parameter for OUTFILE :
           DISP=O   if output file exists, then overwrite (default)
           DISP=A   if output file exists, then append, else create
           DISP=N   if output file exists, then abort job (same as JCL's DD DISP=NEW)

  7 - New 'OUTREC' parameter :  records written to the output file may be rebuilt
      from the input record, all or some fields, with a different order

  8 - Bunch of new options :

         --activation-key=your-activ-key   # if you don't want to 'install' hxsm
         --keep-order                      # if you want a Non-Destructive sort
         --throw-empty-records             # as you can expect ...


B) Bugs fixed:
===================

   1) Option '-q' (new fashion : '--quiet') will cause XSM to exit silently
      with a return code, whatever the cause of the error.

   2) INCLUDE/EXCLUDE expressions containing constant C' '  (a blank) :
      This caused stopping expression analysis.
      This is now fixed

   3) Fixed files : in some cases, XSM was writing the output file with blank records.
      This is now fixed




C) New syntax for options and command line switches :
=====================================================

 +--------------------------------------+---------------------------+------------------------------------------+
 | Inside a Parmfile                    | Old switch                |  New switch                              |
 |                                      |                           |                                          |
 |  SORT  FIELDS=(s1,l1,typ1,A/D, ...)  | -ks1,l1[,A/D[,typ1] -k... | [--sort] --key=s1,l1[A/D[,typ1] --key=...|
 |                                      |                           |                                          |
 |  MERGE FIELDS=(...)                  | -m -k...  -k...           |  --merge --key=.... --key=...            |
 |                                      |                           |                                          |
 |  RECORD RECFM=x,LRECL=nnnn           | -rx -lnnnn                |  --recfm=x --lrecl=nnnn                  |
 |                                      |                           |                                          |
 |  OUTFILE name[,RECFM=x[,LRECL=nnnn   | -ooutfile                 |  --outfile=name[,RECFM=x[,LRECL=nnnn     |
 |              [,DISP=O/A/N]]]         |                           |               [,DISP=O/A/N]]]            |
 |                                      |                           |                                          |
 |  INFILE name[,RECFM=x[,LRECL=nnnn]   | name name ...(last parms) |  --infile=name[,RECFM=x[,LRECL=nnnn] ... |
 |                                      |                           |                                          |
 |  SORTWORK dir1[,dir2...]             | -tdir1[,dir2..]           |  --sortwork=dir1 --sortwork=dir2 ...     |
 |                                      |                           |                                          |
 |  STORAGE nnnK/M/G                    | -ynnnK/M/G                |  --storage=nnnK/M/G                      |
 |                                      |                           |                                          |
 |  COLLATE=ASCII/EBCDIC                |  (none)                   |  --collating-sequencxe=Ascii/Ebcdic      |
 |                                      |                           |                                          |
 |  OMIT DUPLICATE KEYS                 | -uk                       |  --unique-key                            |
 |  OMIT DUPLICATE RECORDS              | -ur                       |  --unique-record                         |
 |                                      |                           |                                          |
 |  INCLUDE COND=(s,l,op,const...)      |  (none)                   |  --include=s,l,op,const[,AND/OR,...]     |
 |  EXCLUDE COND=(s,l,op,const...)      |  (none)                   |  --include=s,l,op,const[,AND/OR,...]     |
 |                                      |                           |                                          |
 |  IOERROR IGNORE                      |  (none)                   |  --throw-empty-records                   |
 |                                      |                           |                                          |
 |  OPTION  COPY                        |  (none)                   |  --copy                                  |
 |                                      |                           |                                          |
 |  OPTION  NORUN                       |  (none)                   |  --norun                                 |
 |                                      |                           |                                          |
 |  OPTION KEEP_ORDER (*)               |  (none)                   |  --keep-order                            |
 |                                      |                           |                                          |
 |  OUTREC FIELDS=(fr_src,len,to_dst,   |  (none)                   |  --outrec=fr_src,len,to_tgt,type,...     |
 |    (*)         typ[,fr_src,len,...)) |                           |                                          |
 +--------------------------------------+---------------------------+------------------------------------------+

    (*) : new in version 6.5x



D) Sort/Merge Keys :
====================

    FIELDS=(start,len,type,A/D,...) or --key=start,len,A/D,type --key=...

    where type may be :

         C  or  CH  :  Bytes, value 0..255 per byte
         I          :  alpha-numeric field, mixed, where case (upper, lower) must be ignored
         B  or  BI  :  same as CH in this version, but to avoid :
                       in future versions, BI may be used for binary values in half-words, words, double-words.
         Z  or  ZD  :  'Zoned' decimal, from COBOL 'PIC S99999... USAGE DISPLAY'
                       set of ascii digits '0'..'9', the last digit holds the sign, and is an ascii char.
         P  or  PD  :  'Packed' decimal, from COBOL 'PIC S99999... USAGE COMP-3'
                       set of BCD pairs of digits, the last pair holds the last digit and the sign.
         N          :  'Numeric' field, fixed length, with leading blanks and a floating (optional) sign '+/-'

E) Microfocus 'variable' files :
================================

    An internal Microfocus 128 bytes header, plus a set af records.
    Each record is prefixed by a 2-bytes length (Big Endian, 3xxx), 
    and is padded to an even number of bytes

    Note : fields are defined from col. 1 = first significant bytes of the record (2-bytes length is dropped)

    Those files are processed with RECFM=M  (max length=4095)


F) Support of mixed files formats and logical records lengths :
===============================================================

     For each input file or for the output file, you can specify its own record format (RECFM),
     and its own logical record length :

        INPFIL pathname,RECFM=F/T/M,LRECL=nnnn       # parmfile
        --infile=pathname,RECFM=F/T/M,LRECL=nnnn     # command line

        OUTFIL pathname,RECFM=F/T/M,LRECL=nnnn       # parmfile
        --outfile=pathname,RECFM=F/T/M,LRECL=nnnn    # command line

     You can also define a default record format and a default logical record length :

        RECORD          RECFM=F/T/M,LRECL=nnnn       # parmfile
        --recfm=F/T/M   --lrecl=nnnn                 # command line

     Rules :

         if RECFM and/or LRECL is specified for a file, then they are taken in account as expected,

         else

         if there is a default RECFM and /or a default LRECL, then they are taken in account as expected,

         else

         default will be RECFM=T,LRECL=250
         


G) 'OUTREC' parameter :  
=======================

        This parameter allows building output records with selected fields in the input record.

        OUTREC FIELDS=(start_in,length,start_out,type[,start_in...])  # parmfile
        --outrec=start_in,length,start_out,type,...                   # command line

        where :

            start_in : offset (from 1) in the input record
            length   : field length, in byte
            start_out: offset (from 1) in the output record
            type     : ignored, compatibility purpose

       examples :

            Building a 30 bytes output record with contiguous fields from cols. 12-14, 1-10, 14-30
             from input record :
             
               OUTREC FIELDS=(12,3,1,C,1,10,4,C,14,17,14,C)      # parmfile
               --outrec=12,3,1,C,1,10,4,C,14,17,14,C             # command line

            Building a 80 bytes output record with non-contiguous fields from cols. 12-14, 1-10, 14-30 
             from input record :

             
               OUTREC FIELDS=(12,3,1,C,1,30,4,C,14,17,64,C)      # parmfile
               --outrec=12,3,1,C,1,10,4,C,14,17,14,C             # command line
            Note : 'holes' will be padded with blanks.


H) Non-destructive sort :
=========================

         For each set of output records sharing the same sort keys, the original input order will be preserved.

         This is done internally by adding the input record number as the last ('minor') sort key.

              OPTION KEEP_ORDER                                  # parmfile
              --keep-order                                       # command line

         Note : this option is NOT compatible with '--unique-record', or '--unique-key'

                                                                 
K) Activation key  :
====================

       If you don't want to 'install' the hxsm program once for all using the supplied 'hhnsinst',
       you may specify your activation at each run time :

            OPTION KEY=XXXX-XXXX-XXXX-XXXX-K                   # parmfile
            --activation-key=XXXX-XXXX-XXXX-XXXX-K             # command line

       This feature will be usefull when the XSM API will be released (soon ...), to run home-brewed
       sort/merge applications on different workstations.


L) Multitasking ('Multi-threading') :
=====================================

       This feature is usefull when sorting (not merging nor copying) large files.

       Its purpose is to 'parallelize' the process of reading the input files by chunks,
       then sorting and writing each chunk onto a sortwork file.

       It gives pretty good performances when several sortwork physical drives are available.

       XSM launch one subtask ('thread') per sortwork directory available, when specified.

       If you have only one sortwork directory, or did not specify any (default = current directory),
       XSM will NOT start multitasking.

       You can force multitasking (for your own curiosity satisfaction) by specifying :

              OPTION PROCS=n                                     # parmfile
              --procs=n                                          # command line

       where "n" is the the number of subtasks that XSM will launch (n=2..4 are good numbers)

       When physical drives are managed  by 'SAN-like', 'RAID-2', or 'VG/LV' subsystems, it may work well.


M) Following soon :

      API (Application Programming interface)
      Support of C-Isam Files


    Feedback is welcome (support@hhns.fr)


HHNS XSM : Extended Sort/Merge Program, PROfessionnal version multi-threading (V6) PROfessionnal version (V5) Basic version (V4) Current versions: V5.30 (Pro) / V4.70 (Basic) - Fev 2006 Supported platforms : IBM PowerPC - AIX 5.3 : 32/64 bits Intel Pentium - Linux 2.6 (FC4) : 32 bits kernel=2.6.11 glibc=2.3.5 Intel Pentium - Linux 2.6 (FC2) : 32 bits kernel=2.6.5 glibc=2.3.3 Intel Pentium - Linux 2.4 (RH9) : 32 bits kernel=2.4.11 glibc=2.3.2 Intel Pentium - WXP/2K : 32 bits IBM zNNNN - zLinux 2.6 (Suse9) : 32/64 bits kernel=2.6 glibc=2.3.3 Bug fixed: =============== EXCLUDE/INCLUDE : last block of data was dropped when recfm=F ---------------------------------------------------------------------------- Release Notes for Version 4.66 / 5.26 : 30-Sep-2005 Supported platforms : Linux-x86 - kernel 2.2, glibc 2.2.4 (RedHat 7.2) (==== should run on ALL linux-i386 versions ===) Linux-x86 - kernel 2.4, glibc 2.3.2 (RedHat 9) Linux-x86 - kernel 2.6, glibc 2.3.3 (Fedora Core 2) Win32 - NTFS (NT, 2K, XP) and FAT32 (NT, 2K, XP, 95, 98) AIX-powerpc-4.3 32-bits AIX-powerpc-5.3 32-bits AIX-powerpc-5.3 64-bits zLinux-S390 32-bits - kernel 2.6, glibc 2.3.3 (Suse 9) zLinux-S390 64-bits - kernel 2.6, glibc 2.3.3 (Suse 9) Sunos-sparc-5.8 32 bits improved OUTFIL parameter (for compatibility with some SORT feature on Main Frames) : 1) Usage : you want to get multiple output files, depending on a selection in the final output stream; 2) Syntax : OUTFIL FILE=nn,INCLUDE=(startcol,length,type,operator,value) or OUTFIL FILE=nn,EXCLUDE=(startcol,length,type,operator,value) or OUTFIL FILE=nn,INCLUDE=ALL where : nn : 01 .. 99 = output file number, to be appended to the ddname 'SORTOF' ( = system variable 'SORTOF') startcol,length : field definition type : always 'CH' operator : one of 'EQ' 'NE' 'GT' GE' 'LT' 'LE' value : C'...' = constant to be matched against the field (You indeed may have several occurences of the same output record in different output files) 3) Example : UNIX : SORTIN="/indir/the_big_file" ; export SORTIN SORTOF1="/outdir/code_01" ; export SORTOF1 SORTOF2="/outdir/code_02" ; export SORTOF2 SORTOF99="/outdir/code_03_99" ; export SORTOF99 Windows : set SORTIN=D:\indir\the_big_file.dat set SORTOF1=E:\outdir\code_01.dat set SORTOF2=E:\outdir\code_02.dat set SORTOF99=E:\outdir\code_03_99.dat Parm file (all Oper.Systems) : SORT FILEDS=(1,10,CH,A) ; --- sort input by costumer number RECORD RECFM=V,LRECL=250 INPFIL DD:SORTIN ; --- define input file as a 'ddname' ; --- define 3 output files, depending on zip code col.37-38 OUTFIL FILE=1,INCLUDE=(37,2,CH,A,EQ,C'01') ; = ddname SORTOF1 OUTFIL FILE=2,INCLUDE=(37,2,CH,A,EQ,C'02') ; = ddname SORTOF2 OUTFIL FILE=99,INCLUDE=(37,2,CH,A,GE,C'03') ; = ddname SORTOF3 Run XSM (all Oper.Systems) : hxsm526 -v myparms.xsm The result will be ordered by col. 1-10 (for example, a costumer number), and divided into 3 files, depending on col. 37-38 (for example, a zip code) - 1st file ('code_1') will hold all records with code '01' - 2nd file ('code_2') will hold all records with code '02' - 3rd file ('code_03_99') will hold all other records. WARNING : IN THIS VERSION, THE OPTION "FILE=01 [02 .. 09]" WILL BE HELD AS "FILE=1 [2 .. 9]" SO, LET YOU DEFINE THE VARIABLES AS 'SORTOF1', 'SORTOF2' ... 'SORTOF9' AND NOT AS 'SORTOF01', 'SORTOF02' ... 'SORTOF09' ---------------------------------------------------------------------------- Release Notes for Version 4.65 / 5.25 : 01-Jul-2005 Bug fixed: internal loop during merge operation on RECFM=F files New OPTIONS subparameter: OPTIONS NORUN This will prevent XSM to really do the job. Use it to check your parmfile. Change in the format of the VFIELDS Parameter : VFIELDS=.....,FIELDSEP=.... The FIELDSEP Subparameter value can be a symbolic name, or an hexadecimal value Symbolic names accepted : BAR = the '|' char TAB = the Tabulation (X'09') char COMMA = the ',' char COLUMN = the ':' char DIARESIS = the '#' char SLASH = the '/' char BACKSLASH = the '\' char SEMICOLUMN or SEMI-COLUMN = the ';' char SINGLEQUOTE or SINGLE-QUOTE = the "'" char DOUBLEQUOTE or DOUBLE-QUOTE = the '"' char Hexadecimal value : X'hh' (UPPERCASE X, SINGLEQUOTE, 2 Hex digits, SINGLEQUOTE) Examples : VFIELDS=(.....),FIELDSEP=SEMI-COLUMN VFIELDS=(.....),FIELDSEP=X'7C' By default, the FIELDSEP value is TAB (X'09') ---------------------------------------------------------------------------- Release Notes for Version 4.59 / 5.19 : 18-Nov-2004 Performances enhanced for large files (> 10GB) Bugs fixed: - INCLUDE/EXCLUDE/OMIT : new format ex : INCLUDE COND=(1,5,CH,EQ,C'AB CD',AND,21,2,LE,X'F2B3') EXCLUDE COND=(11,6,CH,EQ,C'XYZ AB') (note : previous format is obsolete) - when binary files (RECFM F) are piped to hxsm's standard input, a premature EOF was set on UNIX platforms ---------------------------------------------------------------------------- Release Notes for Version 4.57 / 5.17 : 18-Jul-2004 Bugs fixed: - outfile not written when input file size = storage - epsilon ---------------------------------------------------------------------------- Release Notes for Version 4.56 / 5.16 : 26-Aug-2004 Several bugs fixed: - 'out of memory' when sorting 'worst case' files (All systems). - Seek error on some very large files (W32 only) - improved performances on very large files (All Systems) ---------------------------------------------------------------------------- Release Notes for Version 4.55 / 5.15 : 14-May-2004 1) the documentation is updated (www.hhns.fr/products/xsm/fr/xsmqref.php www.hhns.fr/products/xsm/en/xsmqref.php) 2) New OPTION statements OPTION COPY INCLUDE/EXCLUDE COND=(cond) (see below for '(cond)' ) 3) New OUTFIL statement OUTFIL FILE=1,INCLUDE/EXCLUDE=(cond) OUTFIL FILE=2,INCLUDE/EXCLUDE=(cond) ... with : FILE=n : the program will search an environment variable ('DDNAME') of the form : SORTOFn=outfile_name (i=1,2,...) and (cond) looking like : (start,len,CH,OP,C'aaa...aaa',AND/OR,cond.... ) with 'OP' = one of EQ/NE/LT,LE,GT,GE 4) bugs fixed: options OMIT DUPLICATE RECORDS/KEYS did not drop the first duplicate record/key ---------------------------------------------------------------------------- Release Notes for Version 4.51 / 5.11 : 25-Nov-2003 1) the documentation is updated (www.hhns.fr/products/xsm/fr/xsmqref.php www.hhns.fr/products/xsm/en/xsmqref.php) 2) New OPTION statement OPTION COLLATING=EBCDIC : will respect EBCDIC Collating sequence for : - lower-case - upper-case - digits 3) Bug fixed: Bad sort order for packed decimal fields when sign is xF instead xC or xD ---------------------------------------------------------------------------- Release Notes for Version 4.50 / 5.10 : 24-Aug-2003 1) the documentation is updated (www.hhns.fr/products/xsm/fr/xsmqref.php www.hhns.fr/products/xsm/en/xsmqref.php) 2) New parameters for the SORT/MERGE statement a) SORT FIELDS=all : will sort the entire records (no key definition) (you must provide max len. record in LRECL=...) b) SORT FIELDS=(start1,len1,A/D,start2,len2,A/D,...),FORMAT=BI (if all fields are of the same type : BI, CH, ...) 3) DDNAME for INPFIL/OUTFIL : the filenames in INPFIL/OUTFIL statements can be given as 'DD:somename'. XSM will get the actual filename via the corresponding environment variable. Ex : INPFIL DD:SORTIN1,DD:SORTIN2 OUTFIL DD:SORTOUT those two statements will cause XSM to search the actual filenames via a 'getenv' for the environment variables "SORTIN1", "SORTIN2", "SORTOUT" ---------------------------------------------------------------------------- Release Notes for Version 4.48 / 5.08 : 19-May-2003 The version 5.08 can sort files larger than 2/4 GB. The STORAGE parameter is no more needed : it is computed dynamically, at best. But when sorting very huge files ( > 64 GB ), it is safe to give a STORAGE parameter less or equal to (total real storage - (64MB or 128MB)), in order to avoid heavy paging. ---------------------------------------------------------------------------- Release Notes for Version 4.45 / 5.05 : 04-Feb-2003 Correct a bug giving some 'INTERNAL ERRORS' when sorting huge files of small records, with 'IOERROR IGNORE' and keys larger than the smallest record. ---------------------------------------------------------------------------- Release Notes for Version 4.44 / 5.04 : 18-Jan-2003 Correct a bug involving VFIELD's processing ---------------------------------------------------------------------------- Release Notes for Version 4.43 / 5.03 : 03-Jan-2003 Correct a bug in memory allocation when sorting more than one file ---------------------------------------------------------------------------- Release Notes for Version 4.42 / 5.02 : 11-Dec-2002 the 'STORAGE' parameter is computed dynamically. For most jobs, the parameter is not needed. You may override the result with your own 'STORAGE' parameter, in case of very large files to sort, when you think you don't have enough available storage, in order to avoid paging. ---------------------------------------------------------------------------- Release Notes for Version 4. / 5. : Oct-2002 ==> SUPPORT OF FILES LARGER THAN 2GB / 4GB : tested with a 5 GB file on - WinNT/4, sp6 - Win2k, sp1 - Linux 386, Kernel 2.4.13, glibc-6 - Linux 390, Kernel 2.4.17, glibc-6 Note : should run on W9x with a NTFS Driver (www.Winternals.com) V4.33c (apr 2002) - some minor bug corrected for small binary files V4.33b: - some minor bug corrected, that could provoque a crash at initialization V4.33 : - major improvement in handling I/O's - twice faster V4.13 : correct a bug in V4.12 that inserted blank lines in text files. V4.12 some improvement in handling I/O's ================================================================================ Please send bugs reports, or feelings to : support@hhns.fr or mail@hhns.fr Enjoy fast sort. Henri Henault

Back to top