Resolving a dynamic call site inside its own program

Read first: Value sets and dynamic calls, Area need

Area need charges a dynamic call site whose callees it does not know the union of every member's entry need. Where some programs need thousands of bytes, that union is thousands of bytes, and it spreads from the site's program to its callers.

PROGA has three dynamic XCTLs that hand over its commarea. Two take their names from PROGA's own literals, so a walk over PROGA alone resolves them: they are charged 8 and 16 bytes, their callees' rows. The third copies its name out of the commarea and is charged the union of all four members' rows, 32 bytes.

Each site gets one forward reaching-definitions walk over its operand's value set. Terms are in the COBOL/CICS primer.

The example

PROGA's sites are on lines 24, 27 and 30, each handing over its 100-byte commarea. PROGB, PROGC and PROGD each XCTL on to a name in their own commarea: PROGB needs bytes [20, 28) on entry, PROGC [40, 48), PROGD [60, 68). Nothing calls PROGD.

flowchart LR
    S24["PROGA line 24
XCTL WS-FIRST-PGM"] --> B["PROGB
needs [20, 28)"]
    S27["PROGA line 27
XCTL WS-MENU-PGM"] --> B
    S27 --> C["PROGC
needs [40, 48)"]
    S30["PROGA line 30
XCTL WS-NEXT-PGM"] -.->|"callee unknown"| U["every member's row
for the commarea"]
    D["PROGD
needs [60, 68)
nothing calls it"]

Solid arrows end at programs found inside PROGA; the dashed one has no known end. That is the whole answer for this example, what each site is charged:

The lines that matter: each site's operand, and every write to it. LK-NEXT is commarea bytes [0, 8).

* PROGA
01  WS-FIRST-PGM  PIC X(8) VALUE 'PROGB'.
01  WS-MENU-PGM   PIC X(8).
01  WS-NEXT-PGM   PIC X(8).
    IF WS-FLAG = 'Y'
        MOVE 'PROGB' TO WS-MENU-PGM
    ELSE
        MOVE 'PROGC' TO WS-MENU-PGM
    END-IF.
    MOVE LK-NEXT TO WS-NEXT-PGM.

* line 24, WHEN '1'
    EXEC CICS XCTL PROGRAM(WS-FIRST-PGM)
        COMMAREA(DFHCOMMAREA) END-EXEC
* line 27, WHEN '2'
    EXEC CICS XCTL PROGRAM(WS-MENU-PGM)
        COMMAREA(DFHCOMMAREA) END-EXEC
* line 30, WHEN OTHER
    EXEC CICS XCTL PROGRAM(WS-NEXT-PGM)
        COMMAREA(DFHCOMMAREA) END-EXEC

* PROGB
01  DFHCOMMAREA.
    05  LK-PAD   PIC X(20).
    05  LK-TGT   PIC X(8).
    05  LK-TAIL  PIC X(72).
    EXEC CICS XCTL PROGRAM(LK-TGT) END-EXEC.

PROGC and PROGD are the same with LK-PAD 40 and 60 bytes long (and LK-TAIL shortened to keep 100 bytes). Their own XCTLs hand over no area, so area need has nothing to charge them and does not list them among its dynamic sites.

       IDENTIFICATION DIVISION.
       PROGRAM-ID. PROGA.
       DATA DIVISION.
       WORKING-STORAGE SECTION.
       01  WS-FIRST-PGM         PIC X(8) VALUE 'PROGB'.
       01  WS-MENU-PGM          PIC X(8).
       01  WS-NEXT-PGM          PIC X(8).
       01  WS-FLAG              PIC X.
       01  WS-MODE              PIC X.
       LINKAGE SECTION.
       01  DFHCOMMAREA.
           05  LK-NEXT          PIC X(8).
           05  LK-REST          PIC X(92).
       PROCEDURE DIVISION.
       MAIN-PARA.
           IF WS-FLAG = 'Y'
               MOVE 'PROGB' TO WS-MENU-PGM
           ELSE
               MOVE 'PROGC' TO WS-MENU-PGM
           END-IF.
           MOVE LK-NEXT TO WS-NEXT-PGM.
           EVALUATE WS-MODE
               WHEN '1'
                   EXEC CICS XCTL PROGRAM(WS-FIRST-PGM)
                       COMMAREA(DFHCOMMAREA) END-EXEC
               WHEN '2'
                   EXEC CICS XCTL PROGRAM(WS-MENU-PGM)
                       COMMAREA(DFHCOMMAREA) END-EXEC
               WHEN OTHER
                   EXEC CICS XCTL PROGRAM(WS-NEXT-PGM)
                       COMMAREA(DFHCOMMAREA) END-EXEC
           END-EVALUATE.

       PROGRAM-ID. PROGB.
       ...
       LINKAGE SECTION.
       01  DFHCOMMAREA.
           05  LK-PAD           PIC X(20).
           05  LK-TGT           PIC X(8).
           05  LK-TAIL          PIC X(72).
       PROCEDURE DIVISION.
       MAIN-PARA.
           EXEC CICS XCTL PROGRAM(LK-TGT) END-EXEC.

In cobble's node names the three sites are MAIN-PARA/2.when.0.0, MAIN-PARA/2.when.1.0 and MAIN-PARA/2.when.2.0. The operands are WS-FIRST-PGM@working_storage[0:8], WS-MENU-PGM@working_storage[8:16] and WS-NEXT-PGM@working_storage[16:24].

Step by step

Each site gets its own walk, over its own operand. The three walks are shown side by side because they visit the same nodes. Each row is the set arriving at that node, which is what the walk reports for a site. Values are shown as stored: PIC X(8) pads 'PROGB' with three spaces, written ␣. The program has no loop, so the worklist visits each node once, in order.

The --sites file for the same run holds one row per dynamic site, with the operand, the result, the reasons, and every write in the program that touches the operand. The VALUE clause is not a write, so line 24 lists none.

{"member":"PROGA","resolved":true,"place":"MAIN-PARA line 24","transfer":"XCTL",
 "operand":"WS-FIRST-PGM@working_storage[0:8]","callees":["PROGB"],"reasons":[],
 "definitions":[]}
{"member":"PROGA","resolved":true,"place":"MAIN-PARA line 27","transfer":"XCTL",
 "operand":"WS-MENU-PGM@working_storage[8:16]","callees":["PROGB","PROGC"],"reasons":[],
 "definitions":[
  {"place":"MAIN-PARA line 17","kind":"literal","values":["PROGB   "],"sources":[],"reasons":[]},
  {"place":"MAIN-PARA line 19","kind":"literal","values":["PROGC   "],"sources":[],"reasons":[]}]}
{"member":"PROGA","resolved":false,"place":"MAIN-PARA line 30","transfer":"XCTL",
 "operand":"WS-NEXT-PGM@working_storage[16:24]","callees":[],"reasons":["inbound-linkage"],
 "definitions":[
  {"place":"MAIN-PARA line 21","kind":"copy","values":[],"sources":["LK-NEXT@linkage[0:8]"],
   "reasons":["inbound-linkage"]}]}

Other writes, one program each

Small programs, each ending in XCTL PROGRAM(WS-PGM) COMMAREA(WS-AREA) unless noted, run through the same stage. The reasons column is the site's; for the copy, the write's own reasons say something different, which is explained under the rule.

Why literals settle it

Steps 3 and 4 of the first walk decided line 27. Here is the part of PROGA's graph they covered, with the set of WS-MENU-PGM, line 27's operand, on each edge:

flowchart TD
    E["entry"] -->|"{} + uninitialised"| I["line 16
IF WS-FLAG = 'Y'"]
    I -->|then| T["line 17
MOVE 'PROGB'"]
    I -->|else| F["line 19
MOVE 'PROGC'"]
    T -->|"{'PROGB␣␣␣'}"| J["line 21
MOVE LK-NEXT TO WS-NEXT-PGM"]
    F -->|"{'PROGC␣␣␣'}"| J
    J -->|"{'PROGB␣␣␣', 'PROGC␣␣␣'}"| V["line 22
EVALUATE WS-MODE"]
    V --> S24["line 24 site"]
    V --> S27["line 27 site
resolved: PROGB, PROGC"]
    V --> S30["line 30 site"]

In step 3 each arm overwrites all eight bytes with a literal, so it replaces what arrived, the uninitialised doubt included. In step 4 the arms meet at line 21, so the set is their union; line 21 writes WS-NEXT-PGM, not WS-MENU-PGM. Every value reaching line 27 is a literal, so the site is resolved. Line 21's copy is also why step 7 leaves line 30 unresolved: it gives WS-NEXT-PGM inbound-linkage.

That is the whole test. If every value reaching a site is a literal this program wrote, the site is resolved here. If a copy reaches it, the name may have come from a caller, a callee or a file, which this program alone cannot see, and the site goes to the cross-program walk.

One doubt is let through: uninitialised. In the fragment below there is no ELSE, so the uninitialised path reaches the site. That storage belongs to this program, and no other program can have put a name in it. The site can reach only the names its literals spell, as cobble reports for it:

       01  WS-PGM               PIC X(8).
       ...
           IF WS-FLAG = 'Y'
               MOVE 'PROGB' TO WS-PGM
           END-IF.
           EXEC CICS XCTL PROGRAM(WS-PGM)
               COMMAREA(DFHCOMMAREA) END-EXEC.

What one unresolved site costs

Area need works out which bytes of a handed-over area each program must keep alive for the fixpoint: at a call site that hands over an area, whatever the callee needs on entry. Step 2 of the second walk charged lines 24 and 27 only their callees' rows. Line 30's dashed arrow has no known end, so the only safe charge is the union of every member's entry row for the commarea, PROGD's included, though nothing reaches PROGD. Step 3 then shows all 32 bytes in PROGA's own entry row, from line 30 alone.

Without local resolution, all three of PROGA's arrows would be dashed, and every program with even one dashed arrow pays the union. So: for each dynamic site, can its solid arrows be drawn from its own program alone? If yes, charge only the programs at their ends.

Where this shows up in reverse engineering

On a large CICS corpus (about 3,800 programs and 19,000 jumps), the only goal was a call graph with the dynamic targets filled in. Those targets came in three shapes, and PROGA has one of each; the line numbers are those of the full listing.

Only the third row, about 5% of the corpus's sites, needs the cross-program walk: a caller put the next program's name in the commarea. The area-need stage was charging every dynamic site as if it were the third row. Answering rows 1 and 2 locally stops that, so a program's row reflects the programs it really reaches.

The rule

Each line of the box has already run once in the walks above. initial(o) is step 1 of the first walk: WS-FIRST-PGM starts at its VALUE, the other two at uninitialised. in(n) as a union is step 4, where the two arms meet at line 21. The replaces line is step 3 (lines 17 and 19); the otherwise line is step 4's copy, which leaves WS-NEXT-PGM with inbound-linkage. resolved(site) is steps 5 to 7, and charge is step 2 of the second walk.

The last line has two edges the walks do not reach. Two one-site variants of PROGA, run through cobble with PROGB, PROGC and PROGD beside them, show them; each ends in XCTL PROGRAM(WS-PGM) COMMAREA(DFHCOMMAREA), and DFHCOMMAREA is PROGA's 100 bytes.

Row 1 is a callee outside the corpus, which is charged every byte of the area, as is one that does not receive that slot. Row 2 is a site resolved to nothing, which is charged nothing.

Why uninitialised is accepted, and a copy is not

Uninitialised means one thing here: on some path, no write reached the operand and no VALUE covers it. In the IF-without-ELSE fragment under why literals settle it, that path is the one where WS-FLAG is not 'Y'. That storage is this program's own working storage. No caller, callee or file put anything in it. So the only names the site can reach are the literals on the other paths, there PROGB. If that path really runs, the operand holds whatever the storage started as, not a name some other program chose.

A copy is different. This fragment is plainly PROGB:

       01  WS-PGM               PIC X(8).
       01  WS-OTHER             PIC X(8).
       ...
           MOVE 'PROGB' TO WS-OTHER.
           MOVE WS-OTHER TO WS-PGM.
           EXEC CICS XCTL PROGRAM(WS-PGM)
               COMMAREA(DFHCOMMAREA) END-EXEC.

The writes are computed once per program, without a state, so the second MOVE above looks up WS-OTHER in an empty state and finds it "unwritten". That is a fact about the empty state, not about the program: WS-OTHER holds 'PROGB' by then here, and elsewhere might hold a name read from a file. The walk does not follow the copy to find out.

cobble's --sites row for it is below; its line numbers are those of the full variant program. The copy's own reasons still show uninitialised, but the walk drops that reason, so the write is not-a-literal and so is the site. Accepting it would resolve the site to nothing and charge nothing, while the combine, following the copy, links it to PROGB.

{"member":"PROGA","resolved":false,"place":"MAIN-PARA line 15","transfer":"XCTL",
 "operand":"WS-PGM@working_storage[0:8]","callees":[],"reasons":["not-a-literal"],
 "definitions":[
  {"place":"MAIN-PARA line 14","kind":"copy","values":[],
   "sources":["WS-OTHER@working_storage[8:16]"],"reasons":["uninitialised"]}]}

A copy whose source carries a real reason keeps it. The table lists them with where on this page each shows up.

What goes wrong

Each case below is a small variant of PROGA, run through cobble beside PROGB, PROGC and PROGD. Where PROGA itself reads none of its commarea, the union an unresolved site pays is their three rows, [20, 28) [40, 48) [60, 68): 24 bytes.

• One unresolved site is enough. In the example, two of three sites resolved, and PROGA's entry row is still 32 bytes, because line 30 alone pulls in the union. Delete the WHEN OTHER branch (lines 29 to 31) and run again:

  PROGA	entry, commarea
  as written	[0, 8) [20, 28) [40, 48) [60, 68), 32 bytes
  without line 30	[20, 28) [40, 48), 16 bytes

  Local resolution shrinks a row only when it resolves every unresolved site that hands over that area. With line 30 gone, LK-NEXT no longer feeds a target either, so [0, 8) goes too.
• A copy of a literal stays unresolved. The MOVE 'PROGB' TO WS-OTHER, MOVE WS-OTHER TO WS-PGM fragment above is plainly PROGB, but the walk tracks one operand and does not follow copies. The site pays the union. The later cross-program walk does follow it.
  unresolved: not-a-literalcharged 24 bytesPROGB alone: 8 bytes
• Only one exact extent. An operand that could be several places, such as a subscripted WS-PGMS(WS-I), gets operand-not-one-exact-extent and stays unresolved here, even though both slots hold literals, and even though the combine can resolve it.

         01  WS-TABLE.
             05  WS-PGMS          PIC X(8) OCCURS 2 TIMES.
         01  WS-I                 PIC 9.
         ...
             MOVE 'PROGB' TO WS-PGMS(1).
             MOVE 'PROGC' TO WS-PGMS(2).
             EXEC CICS XCTL PROGRAM(WS-PGMS(WS-I))
                 COMMAREA(DFHCOMMAREA) END-EXEC.

  unresolved: operand-not-one-exact-extentcharged 24 bytes
• Conditions are not evaluated. A literal on a branch that never runs still counts. Below, nothing writes WS-NEVER, so the MOVE 'PROGC' never runs, yet the site is resolved to PROGB and PROGC and charged PROGC's row too. That is loose, not wrong.

         01  WS-PGM               PIC X(8) VALUE 'PROGB'.
         01  WS-NEVER             PIC X VALUE 'N'.
         ...
             IF WS-NEVER = 'Y'
                 MOVE 'PROGC' TO WS-PGM
             END-IF.
             EXEC CICS XCTL PROGRAM(WS-PGM)
                 COMMAREA(DFHCOMMAREA) END-EXEC.

  resolved → PROGB, PROGCcharged 16 bytesPROGB alone: 8 bytes
• Values that are not program names. If no value could name a program (nothing left once trailing spaces are trimmed, or an unprintable character), the site is resolved to nothing and charged nothing, with no-program-name: row 2 of the table under the rule. That matches the combine, which links such a site to nothing.
• The walk starts at the program's entry. It assumes storage holds its VALUE or nothing on entry. A CALLed batch subprogram keeps its working storage between calls. Below, the site runs before the MOVE, so the walk sees only uninitialised there; on a second call, WS-PGM already holds 'PROGB' from the first. The value is still one of this program's literals, but the site's callees miss it.

         01  WS-PGM               PIC X(8).
         ...
             CALL WS-PGM USING DFHCOMMAREA.
             MOVE 'PROGB' TO WS-PGM.
             GOBACK.

  arriving: {} + uninitialisedresolved → nothingreasons: uninitialised, no-program-name
• Without a CSD, transactions say so. A RETURN TRANSID or START without --config is unresolved with no-csd, not resolved to nothing: row 2 of the table below. Resolving to nothing would charge nothing and drop the bytes the started program needs.
• An incomplete CSD drops bytes. The same argument applies to a transaction id the CSD does not list, but there the site is resolved to nothing and charged nothing, with transaction-not-in-csd: row 3. If the CSD is merely incomplete and ZZZZ does start a program, that program's need is dropped. This is unlike a callee missing from the corpus, which is charged every byte (row 1 of the table under the rule). The tool trusts the CSD it is given; the --sites reason is the only trace.

The two CSD cases, as one site: RETURN TRANSID(WS-TRAN) COMMAREA(DFHCOMMAREA), with a CSD whose only line is DEFINE TRANSACTION(TRNB) PROGRAM(PROGB) GROUP(TEST).

In cobble

The resolution runs inside cobble-dataflow-area-need, once for each member before the rounds. One graph is built per program, and each dynamic site gets its own walk over it, as PROGA's three sites do in the first walk above. The answers split into resolved sites and unresolved ones, the solid and dashed arrows of the graph under the example. A resolved site is charged its callees' rows; an unresolved one, the union of every member's entry need. Transaction ids from RETURN TRANSID and START are mapped to programs through the CSD files of the corpus config named by --config.

PROGA's --sites rows and charges above came from these commands, run over PROGA to PROGD with the bridge environment sourced and the corpus config listing the four programs:

cobble-parse-corpus --config corpus.toml
cobble-extents --asg-dir asg --out extents.jsonl
cobble-dataflow-area-need --asg-dir asg --extents extents.jsonl \
    --out need.jsonl --sites sites.jsonl --explain PROGA

PROGA's row in need.jsonl is below. entry holds the four ranges of the second walk's last step as flat start, end pairs. sites lists only the two resolved sites, by node name, with their callees; line 30 is not there.

{"member":"PROGA","receives":[["commarea",0]],
 "entry":[[["commarea",0],[0,8,20,28,40,48,60,68]]],"back":[],
 "sites":[["MAIN-PARA/2.when.0.0",["PROGB"]],["MAIN-PARA/2.when.1.0",["PROGB","PROGC"]]]}

--sites writes one row per dynamic site, like the three under step by step, with each write that touches the operand marked as a literal, a copy or other. Sites that a finished fixpoint and combine resolved are added from --fixpoint-dir and --combined-dir. The command is documented under cobble-dataflow-area-need in the field dataflow reference.

Next: rounds across programs, and walking only what changed.