Before showing the porting to the accountant lady, I absolutely need to complete the port of the most important executable, the one used all day long for all the data input.

The situation is the following:

276 function starts
340 BEGIN SEQUENCE
602 RETURN
1096   BEGIN SEQUENCE // nested
1215   END SEQUENCE   // nested
1568 END SEQUENCE
1590 function ends

Several BREAKs are present in different lines of the file.

As in other cases also in this one the SEQUENCE is used for unconditional jump. This is a more complex situation than the previous ones but not too different: I will use the same solution I used to solve error 8 in the previous post. I will convert the RETURN to BREAK 2 and all BREAKs from line 341 to 1095 and from line 1216 to 1567 to BREAK 1. I will then add a RECOVER USING before line 1568 to handle the two cases.

Is it enough? Probably. But there is one possibility that I still don’t handle: BREAK can be used by a function called in the 2 blocks of code listed above! I browsed the code and I didn’t find any external function calls, so my changes should be safe. If a BREAK is somewhere deeper in the code, the RECOVER will fail with an unitialized variable since it’s passed value is NIL.

Now also this executable compiles.

In the previous post I listed the errors I had to correct in order to compile the applications. Error number 6 puzzled me a bit and I asked in the Harbour mailing list.

There is a

SET KEY K_Fx TO myfunc()

but the function in not linked and Harbour linker complains, and Clipper 87 doesn’t.
Since there is a MYFUNC.PRG file, is it compiled automatically by Clipper 87?

The people in the mailing list told me that no, the file is not automatically compiled, linker doesn’t complain but when you press key Fx the program stops and reports a messing function.

I don’t know how Clipper 87 VM works… but the test results are clear.

This made me have a new look at that part of the code and… the program prints in a line on the screen which F keys are active in the form… and this Fx key is not listed! It may be that pressing that keys in the form generates an error in Clipper 87 compiled code !!!!

Thanks to everybody for help and Maurizio for the tests on Clipper 87.

As described in the previous posts, a big number of duplicated functions had me change my mind on how to complete the porting. I’m now back to the original software setup: a batch file that calls several different executables depending on errorlevel values.

I reverted the changes done, starting from removing the source files from exe01.hbp and uncomment the initializazion function calls I disabled previously: the program compiled as described in the previous post (but I still have to solve that RETURN in the SEQUENCE)

Just to streamline the various .hbp files I’m going to create, I decided to have a global hbc files that lists all the use libraries. I could have used a hbmk.hbm file but I prefer to have everything listed.

I created libs.hbc file listing all the libs:

libpaths=.
libs=applib
libs=funcky
libs=mylib

In this way all the libraries don’t need to be listed in each .hbp but I just need to add a reference to libs.hbc. It is not only one line vs 4 in each hbp but something more strategic: in this way it is possible to add other libraries in a centralized place. For example, I’m thinking about the need to add some functions for converting the reports printed on file to PDF, and I may decide to use a library I already have: instead of updating all the .hbp files, I may just add the new one here. I also noticed that the error handlers I saw in the code are not called so I will probably need to add my error handler functions too.

I then created the file all.hbp with all the .hbp included but commented, except the one I already created.

-hbcontainer
applib.hbp
mylib.hbp
funcky.hbp
exe01.hbp
# exe02 is the Utility Menu, included in exe01
exe03.hbp 
# exe04.hbp
...
# exe13.hbp

Now with a single command I can compile everything in a standard, repeatable, way, all of them in one go, with the command:

hbmk2 -rebuildall all.hbp

Notice that the build will stop at the first error and you don’t have to check for errors like in a batch based solution. I think that all projects should have a all.hbp equivalent, a way to build the full project with one command, that can be used in an automatic building system.

I had the first 2 executables compiling, linking and running. The second has been patched but the patch not committed in the repository. I decided that before looking at that code (with the RETURN 300 lines after the BEGIN SEQUENCE and 600 before the END SEQUENCE) I’d better check if there are some other show-stopping problems. So I commented the exe03.hbp line in all.hbp and converted all .lnk files to .hbp, so that all executables are built except for the one I already know has a problem to solve.

Let’s list all the errors I found and how I solved them.

Error 1: RETURN inside BEGIN SEQUENCE/END. Harbour and Clipper 5.x syntax is probably different from Clipper 87: in the past it was possible to have a RETURN inside the sequence. This is probably the easier case as there isn’t a RECOVER and the return is only used at the start, if the files can’t be opened:

// original code
BEGIN SEQUENCE
   IF my_usedbf( .... )
      RETURN
   ENDIF
   ...

I changed to:

// changed code
IF my_usedbf( .... )
   RETURN
ENDIF
BEGIN SEQUENCE
   ...

Error 2: duplicated function names. This is a function clearly called by error handler. There are some of them in a file, but I never found the place where they are setup. I will check later. In this case I added a prefix.

Third program linked.

Error 3: missing function mkdir(). This function is used, in this program, to create a new workarea, and since a workarea is a subdirectory, it clearly needs to create the directory. It is also used elsewhere but before spending time developing and validating the function (with all that error handling the program uses) I’d like to complete the main port. So I created a stub mkdir() that outputs a trace and then exits the program.

Error 4: unexplicate error message by the linker: undefined reference to `HB_FUN(int0_t) static’.
Przemek explained why it happens and how can be solved. In this case the code uses a function named _size(). This function has a dbf file as parameter and it should return the file size. Since it is used together with mkdir() I created a similar stub file.

Fourth program compiled.

Error 5: undefined function, used by licensing code. In this program the licensing code is still active and this is strange: this makes me wonder if this executable is still used… I adapted the code as per other sources where licensing code is disabled.

Fifth and sixth programs compiled.

Error 5 again: another source code with licensing code active.

Several programs compiled.

Error 6: In a executable there are two missing functions reported by the linker. These functions have names that are already used in other parts of the code, in other executables. I checked if I correctly converted the .lnk to .hbp and I did. So, do I miss these functions? Are they duplicated but not present in the code I have? Should they be in the application library? or… or…
Both functions are used in a SET KEY statement and one has a source file with the same name, and the other is included in that file. I don’t have Clipper 87 installed but I feel there is something interesting possibly going on here. I’m going to ask in the newsgroup. In the meantime I’m commenting out the building of this executable.

Error 7: one comma not necessary in a list of variables

 PRIV var1, var2, var3,  // <- note the ending comma!

Executable compiled.

Error 8: 2 RETURNs in BEGIN SEQUENCE. In this case we have a 150 lines function that extracts data from several DBFs and depending on several conditions that show up during data processing, it RETURNs without completing, or BREAKs to skip some calculations but do something with the data found. It’s like having a 10 step process and after each step you may cancel (RETURN) or skip all the following steps (BREAK) and complete the job anyway. BREAK is used as an unconditional jump, without parameters, since it is used only to skip to the end of the sequence block.
My solution is the following: all original BREAKs, the ones that want the program flow to continue are tranformed to BREAK 1, while all RETURNs are transformed to BREAK 2 and finally a RECOVER block is added to properly handle the situation:

BEGIN SEQUENCE
...
IF condition()
//   BREAK
   BREAK 1
ENDIF
...
IF condition()
   // RETURN
   BREAK 2
ENDIF
...
RECOVER USING nValue
   IF nValue == 2 
      RETURN
   ENDIF
END SEQUENCE

And now also this last executable compiled.

Errors 1,2,5,7 and 8 were solved. Errors 3 and 4 are just stubs functions like the others working on directories. For error 6 I need infos from someone with a working Clipper 87. Of course there is still the RETURN in that big function to fully analyze, with nested SEQUENCE blocks, that probably will be solved as per error 8. Unfortunately, this block is in the most important executable, the one used to load the data inside the program… I will do in the next post.

I need to show the accountant lady that the program can be converted and the first round is complete: it works somehow, for sure it needs testing, but I think that there will not be too many problems in the future.

This is the title of chapter 11 from the book “The mythical man-month”, by Frederick Brooks. It is not exactly my case but I always have in mind the concept: you should be ready to change your mind, throw away the job done, and rollback or start from scratch.

You can do interview to users, analyze processes, write stories, you never know how a project will develop. In this case I’m doing a porting of a program written in the late 80s so I need to understand the original programmer workflow. My idea on how to port the program had to be validated spending as little time as possible. Some were succesful, now I’m facing the first one that didn’t, and it is a show-stopping one.

I repeated my idea several times: unify all the executables into one. The idea got a stop when I discovered, merging the first external group of source files, that the original programmer used function with identical names in different sources and that the code inside the functions is different!

Let’s have a look at the lockok() function:

FUNCTION lockok()

   SELECT dbf02
   IF ! my_file_lock( ...  )
      RETURN .F.
   ENDIF

   SELECT dbf05
   IF ! my_file_lock( ...  )
      RETURN .F.
   ENDIF

   ...

   RETURN .T.

It is a really simple code: it tries to put a file level lock on several files. This function is defined in 3 different source files (linked in 3 different executables, of course) with a different list of databases. It is used in places where the programmer wanted to be sure that nobody else could change the data during an update that spans 3 to 8 databases. Unfortunately, if the function reports that non all files could be locked, the program just skips the big update…

We have two easy ways to handle this situation. Both need that you understand to which executable belong the source files that have the function definitions and, more important, the ones where the function is just used. Once you trace them, you can simply prefix the function name with a identifier and change all the calls that belong to that executable, or refactor the function to receive the databases alias as a parameter.

In this case the second option is probably better, so that similar looking code is deduplicated. I also discovered that there are a couple of other functions, with similar sounding names, that perform the same job: probably the programmer was creating duplicates in his own programs..

This is a really simple case, one that can be handled very quickly. But the second is really more delicate. Infact the second duplicate function I analyzed was about 150 lines long and it looked really similar, I’d say that they were perfectly identical… I was about to port the function to the application library but in the end decided to use a diff tool and I finally noticed that a couple of lines were a bit different: 2 out of 150 lines were different in 2 IFs that had 2 expressions vs 3 expressions. I don’t know what the code really does, and if this check is really important to the data I’ll use and I don’t know if I could add it to the missing one. Solution: add a prefix to the function names.

In the previous post I already reported about one function duplicated, different, but used only in the source file they were defined: adding STATIC solved this case.

Another case was a function whose code is completely different but functionally equivalent. Imagine you have to report the sum of the values in a array and use a FOR loop once and a aEval() the other time. But to deduplicate the code I should write test cases to see that there are not edge cases handled differently.

Too many different cases: first 4 checked, 4 different problems and 4 different solutions. And there are 47 to go! Probably it’s time to change strategy. Merging this code I had a couple of compilation problems I solved with a temporary patch: disabling that code! So I decided to step back and fully mimic the original program, compile all different source code trying to clean up compilation errors and in meanwhile try to experiment a robust solution for passing the subdir to use as current working directory.

So I decided to throw away the changes done to deduplicate the functions and to start in a new direction.