We are looking inside the printing code to understand why the report printed on a file has several blank lines, spaces, etc. Two functions are called inside the loop and the first is not the problem: removing it the result doesn’t change.

The loop is simple, it just prints the data, using just one lookup. But just before looping it updates a field in the current data base row…

IF record_lock( FORCE )
   REPL FIELD->FIELD_1 WITH iif( value = 1, FIELD->FIELD_2, variable )
   GOTO RecNo()
ENDIF
UNLOCK

I think there are some flaws in this code. The first one is the GOTO RecNo() that should be not necessary. The other one is more philosophical: here the program is printing the tax filing reports and instead of locking one record at time, I’d lock the whole file, to avoid that someone, during the 2 hour period the printing needed, could change the data. Working on a data file exclusively should speed up the job but at the time it didn’t matter: the dot-matrix printer was the slowest in the chain.

Anyway, I went to the record_lock() function, that was decompiled. Decompilation doesn’t change instruction order: compiler optimizer can. But I don’t even know if Clipper 87 had one.

I opened the source file, searched the function and… oh…. code is… is… a mess !

Just a little snippet, the first few lines:

FUNCTION record_lock( ..., wait, ... )
   PRIVATE row,col,time,video

   row = Row()
   col = Col()
   time = wait
   video = SaveScreen( 21, 0, 23, 79 )
   IF RLock()
      csrput( row, col )
      RETURN .T.
   ENDIF
   DO WHILE .T. // Loop for RLock() retries

What’s wrong here? Well, strictly speaking, this code has been working for 25 years so it can’t be “wrong”. Would you accept this code from a programmer working for you? I won’t.

Not when this function is called in a “tight loop”, looping, in my test case, 15600 times. The function calls 15600 times Row(), 15600 times Col(), assigns 15600 times the time variabile and 15600 times saves a block of the screen. And if the RLock() works at the first try, 15600 times the cursor coordinates are reset… but… where did we move the cursor?

I got it: Maurizio already warned me in the comments that csrput(), a function from Funcky that I rewrote, was not correct.

My version:

FUNCTION csrput( y, x )
   @ y,x SAY ""
   RETURN .T.

Maurizio version:

FUNCTION csrput( y, x )
   SetPos( y, x )
   return nil

Actually Maurizio was right. My version of csrput() did addeall that empty lines in the printing. Since the cursor was always reset to the same position, the number of empty rows was fixed, and a form feed added.

Problem found. Compiled, linked, run a new print test. Now the printing seems ok, except for the totals. Imagine a report “framed” by a 76 columns * 66 rows box printed with | and – symbols. Also the columns are divided by a | between them.

At the end of each page a subtotal is printed. I don’t know if it is the total of the page or the total of the values printed so far, the important is that when opening the printed file I got a strange result.

This is how it looks in vim:

Looking at the source code, the program prints a line with code similar to this:

@ PROW()+1, 5 say "|            |            |"
@ PROW(), 6 say value PICTURE __PICT

@ PRow()+1 means to print on the following line, forcing the printer to go up one line. @ PRow() means to print on the same line… 6 is > of 5, but 6 is < of PCol(). Something strange here… but, hey, what are that strange blu symbols in the file?

Think again at the dot-matrix era. Or better, at the time of the type-writer. How did you write some letters in bold? Or underlined? You just shifted the roll to the right and typed over the previous letters to have a bold result, or typed a sequence of minus symbol after rotating the roll a bit. Actually, you over-typed. And here is the same.

Dot-matrix printers have 2 different commands: one tells the printing head to go back at column 0 (carriage-return) and the other to move the paper one step (line feed). There are many other commands, like form-feed to go to the next sheet of paper (^L) and the ones to set compressed or expanded fonts (or CPI, characters per inch). ^L is present in all reports, font selection just in a few.

So now it should be clear which was the trick used by the programmer: write one line of the box, carriage return (^M), over write the values, then ^M^L, not shown in the image since it is the standard EOL.

When I will write the converter to create a PDF from the printed file I’ll need to handle these control characters.

I think that the printing test is now completed. Test-printing to a file works with very little changes to the printing source code (just one line added) but for the real changes more code is needed.
This test also shown that some library code needs a good refactoring, something I will do in the next post.

I’m not an accountant. I don’t know how to keep a journal of a company. My program, a mix between ERP and CMS, just prints the invoices then exports the values to be imported into the program I’m trying to port. This is the program with the official financial data. And the data from this program is used for tax reports.

The program is – as you know – Clipper 87 code, and it was written to be used with dot-matrix printer: they were the printers available at the time! And for several years the only way to print some tax reports was using dot-matrix paper: you had to buy the paper and to bring it to an office and have it “stamped” with a progressive number, so that it could be visible if, in case of a control, a paper was missing.
Unfortunately I don’t know the details.

I just remember the accounting lady reserving a full day to ready and print these reports: nobody could enter her room and she was carefully attending at the printer and checking that the paper didn’t jam.

Nowadays, you don’t get this kind of paper anymore. But the program still uses it. The solution in the last years was to export data from the program and import them into the new version, that I said in the first post, is written in a new language and uses SQL. And as I said the new version lacks some features and… you know, I’m doing the port just for this reason.

By the way, you can now print on lasers, of course. And you can use A4 paper, but you still need to have it stamped. When you get the paper back, looking at the box, you can see that page 1 is at the top, 2 is the second, all with the stampint towards you. To have the reports printed correctly, you must use a laser printer that can use the paper stacked in that position. And the first time you will surely discovered the problem when it’s too late. The lady had to revert hundreds of pages, since her laser prints on the down-facing size of the paper.

I have the program start and could create the indexes for a workarea. Now it is time to check one of the most important reason to port the program to Harbour: printing.

Looking at the menu, I choosed one option, the one that prints all the financial movements of one year. I don’t know if it is the one to print on the stamped paper but it is long and… it prints something.

My hope was to find that an external library was used, so that I could locate it or a clone in open source, or rewrite it myself. Instead, plain Clipper 87 commands are used:
SET DEVICE TO PRINTER
a series of @ SAY
EJECT
SET DEVICE TO SCREEN
as needed. Everything is plain code, neither an internal library to easy form feed, headings, footers or subtotals is used. Just plain Clipper code. This means that I have to check all the printing functions and apply the same changes to all… probably I will refactor a bit, but I first want to understand clearly how this work and, as I already said, I’d like not to change printing code too much.

Since I never used this kind of printing method, I asked harbour-users newsgroup. Daniele replied with a good answer: set the printer to a file, let the program print, open the file and convert to PDF. While reading the file, look for special chars, the ones used in dot-matrix printers to set character spacing, font width, eject, etc.
I remember to have written some code for this job when I was porting the ERP to Harbour, to create a rendering engine external to the main program, but in the end I converted my library to output to Win32 first and then to haruPdf.

In this case, my idea is to create the printed file using a temporary file name and then call a function to convert it to PDF, without calling an external program (except the PDF viewer).

Let’s test Daniele idea. The diff is quite simple, isn’t it?

Compile, link, run, select the function from the main menu: the function starts with a form asking for from-to filters pre-filled with first and last values present in the database. Confirmed the printing, the program thinks for about 10 seconds and then a message tells me the printing is completed.

With a command prompt I went to the program directory but I didn’t find the file. Current directory is another!! So I went to that directory and found the file. Opened in vim: the heading was ok, the first data line was ok, then 10 empty lines, then one line with some spaces but no data, then the second data line, in a loop. What is happening? Let’s have a look at the code!

The code uses the variable row to keep track to which line to print and it is correctly upped by 1 correctly. There should be another reason.

In the loop there are two functions called: the first one was the emergency button for the tax reports prints: pressing space during printing would stop the printing, so that you could un-jam the paper, if needed. A menu appeared and you could resume when ready. Back at DOS time, the printing was “online”, with the program waiting for the line to be ouput before executin the following instruction. If the printing took 2 hours, the program was locked for two hours!
This function checks inkey() but if it founds nothing, nothing is done with the screen. This function is outdated, there is no reason to keep it so I deleted from the printing loop: one call less.

Then there is another function call. In the next post I will explain which was the problem and other problems found in the printing code.

In part 1 I described that the program was not able to open the database file: the filename was read from a field and its name contained spaces, it needed to be trimmed.

The code was:

file = FIELD->DBFNAME
key1 = FIELD->INDEX1
key2 = FIELD->INDEX2
key3 = FIELD->INDEX3
IF my_netuse( file, ... )
   SET INDEX to &KEY1, &KEY2, &KEY3

My solution was to trim the filename directly in my_netuse():

FUNCTION my_netuse( file, ....)
...
file := alltrim( file )

Then I discovered that the indexes were not in the backups and so I needed to create them, incorporating the first external program into the main one, as decribed in the previous post.

The indexes were correctly created but with spaces in their name. Also the index names, as you can read in the previous code snippet, are read from a database, and so also I must trim them too!

Shortly, I add a trim() for the index file name during creation and then modified the previous code to:

file = trim( FIELD->DBFNAME )
key1 = trim( FIELD->INDEX1 )
key2 = trim( FIELD->INDEX2 )
key3 = trim( FIELD->INDEX3 )
IF my_netuse( file, ... )
   SET INDEX to &KEY1, &KEY2, &KEY3

Now all files are opened correctly, dbf and ntx.

I remember the accounting lady creating the indexes: from the main menu select Utility, then from the Utility menu, select Rebuild indexes and from a pop-up menu (aChoice) select the index to be recreated and finally confirm pressing a F key. The process was streamlined: I never noticed that when the lady selected Utility from the main menu, the program actually exited and a new one was loaded. The nested menus appeared without issues, and pressing exit you could go back, with the menus clearing itself and then going back to main menu executable.

I now partly understand the complex menu code: part of it is to handle screen coherency. Merging all the different programs into one should allow to simplify that code, if I will see fit.

Let’s have a look at the code of the main menu. When the user selects to open the Utility menu, an errorlevel is set and program exits. Looking at the batch file and tracing that errorlevel, I saw that it starts uti.exe. Obviously there is a uti.lnk file, a uti.prg file and several utiXX.prg files.

Uti.prg has some startup code that is a reduced version of that of the main menu source. Infact it skips some checks due to the fact that is a program that should not be called directly, but it is called with the proper working directory set. It also sets some hotkeys (SET KET), loads some default values, etc. I converted all these function calls and commands to remarks.

Uti.prg contains the Utility menu, that has about a dozen options. Some of them work on archiving data (backups, conversions from/to other programs, creating new workareas), others work on screen configuration (fonts, colors) and finally one opens the list of files that can be indexes.

I decided to disable all the options except the index one: I don’t even know if the other options are used!

So I included the following lines into exe01.hbp:

# Utility menu
src/uti
# Reindex
src/uti04

I then modified the main menu code to call uti() function and not setting errorlevel and exiting. Since the menus use ondo() that has the function names as strings, the program compiled and linked without problems.

The missing and unknown function I talked about here was used in the indexing code and I was a bit worried about it. It is used to create indexes in data root dir if the dbf file is present there and not in the subdir. The program runs correctly and the indexes are created. I didn’t check if the indexes are created in the proper directory – I’m satisfied that they are created.

Merging this code was quite easy:
– located the files to be compiled and added them to exe01.hbp
– removed all the init functions in uti.prg
– added call to uti.prg in main menu

There were a couple of issues that needed to be solved. First of all, the screen. The program didn’t remove the Utility menu when exiting: it was main menu job to refresh the screen and simulate keypresses to show the exact menu when starting, depending on the passed parameter. A lot of code full of K_DOWN, ESC, K_RIGTH that is just waiting to be removed when port completed.
Anyway, I just added a pair of saveScreen/restScreen to uti.prg. It was quick but also, perhaps, not the best choice. Perhaps adding this logic to main menu program will solve this problem for all the future imported functions.

I then did a couple of minor changes to the code. Moving all variables to local and modify some variables to fixed size arrays to variable size, filled with aAdd.
The changes were minimal, and probably a waste of time, but I wanted to go deeper into the code. I will probably refactor the code in the future because I want to add the option to rebuild all the indexes at the same time, and not only one by one: now for each index I have to press 3 keys…