Impact of changing MAX SIZE value in analysts

[TushR]

Dear Team,

I have created an analysts which uses TRANDET record with memory of 5.00 MB.

Analyst will detect CONSECUTIVE decline transactions and send an alert on threshold reach. Currently I kept MAX SIZE as 5 MB but if I changed it to 1 MB then what will be the impact? will work as how it is designed?

I can see that analyst is reaching upto 4.5 MB memory utilization so wanted to keep it less as I have configured lots of analysts like this.

Please refer below analyst logic and guide me if it changed its functionality if I changed its size.

SECTION CONFIG
  MAX_SIZE 5.0 MB 
  NUMERIC TXNCOUNT [0] := 0
  NUMERIC ALERTFLAG [0] := 1
END_SECTION
 
SECTION RULE_DEF

! Get all records from TRANDET record and set counter 
! increment the counter if it is declined response code and reset the counter if approve response code

RULE TRAN_COUNT PRIMARY

  RECORD TRANDET
  WHERE ALL
  REFRESH 30 SECONDS
  NODE ( \NI003 ) 
  SORT DESC LOGTIME 
 
!Increment the counter when the transaction is declined
 
  IF @TRANDET.TERMFIID = "VISA" AND @TRANDET.RESPCODE NOT IN {"00","01","000","001"}
        SET TXNCOUNT := TXNCOUNT + 1                 
  END_IF
 
!Reset the counter when the transaction is approved
  
  IF @TRANDET.TERMFIID = "VISA" AND (@TRANDET.RESPCODE = "00" OR @TRANDET.RESPCODE = "01" OR @TRANDET.RESPCODE = "000" OR @TRANDET.RESPCODE = "001")
        SET TXNCOUNT := 0
  END_IF
 
END_RULE


!Check if the counter value is greater than the threshold and if the send alert flag is on (1)
!If the condition is true, set the alert flag to off (0) to avoid the sending of multiple alerts

RULE SEND_ALERT PRIMARY

  RECORD PNODES
  WHERE ALL
  REFRESH 30 SECONDS
  NODE ( \WHOUATPVAPP01 )

  IF TXNCOUNT > 19 AND ALERTFLAG = 1
        EXEC          #DISPMAN 0100 IMMEDIATELY
        SET ALERTFLAG := 0
        SET TXNCOUNT := 0
  END_IF

END_RULE
 
 
!Rule to reset the send alert flag to on (1) and reset the counter to 0 after x seconds/minutes 
!x seconds/minutes refresh has to match the refresh interval in the rule above
 
RULE RESET_ALERT_FLAG PRIMARY
 
  RECORD PNODES
  WHERE ALL
  REFRESH 30 SECONDS
  NODE ( \NI003 )
 
  ACTION
        SET ALERTFLAG := 1
        !SET TXNCOUNT := 0
  END_ACTION
 
END_RULE
 
END_SECTION
 
SECTION MSG_TEXT
 MSG 0100  "DISPREQ CMD_SRC VISAIssConsDenial MSG_CODE_NUM 100 MSG_DEST PRGTEAM MSG_TITLE Total @TXNCOUNT@ consecutive declined transactions from VISA issuing. MSG_TEXT Analyst detect @TXNCOUNT@ consecutive declined transactions from VISA issuing."
END_SECTION

Thanks in advance,

Tushar Devkar

[Leon_Arens_FNNI]

We do not set this value in our analysts, and the job record shows that each of our analysts are taking from 36 ro 46 meg of memory.  Doing a PSTATE on the PPDs of these processes show that most of the memory is in heap memory. See example below. 

Does the MAX_SIZE param limit the HEAP size or prevent processing of an analyst if the memory limit is reached?

Process memory values and other counters.
    PFS        Current 56248, Size 67108864, Max 110808
    Main Stack Origin  x70000000,  Size 262144, Max 2097152
    Priv Stack Origin  x6DFE8000,  Size 131072, Max 950272
    Heap       Origin  x086E1000,  Size 40235008, Max 1603399680
    Globals    Origin  x08000000,  Size 7213056
    KMSF Guarantee     0
    Resident memory pages   2839
    Messages   Sent 1524393, Received 14914366
    $RECEIVE   Current QLength 0
    Page faults        10144

[ChristopherS]

@TushR 

Hopefully Leon answered your question but it is basically another kill switch similar to the memory limits you set in the Prognosis Subsys.   See this in the help

On your analyst itself..

1) Generally doing anything on Trandet at any volume can be highly inefficent and when you do you need to add some additional throttling and controls. 

I would at minimum if you are going to use Trandet

• Add the where clause to using Log.Time and Current Time to ensure you are only looking at the past 30 seconds.. you don't have this right now and right now the analyst would be scanning more than you want. 

• LOGTIME >  ( CurrentTime - 30 seconds )

• Add a LIMIT clause to your SORT to only scan a set number of transactions.

2) I did a very quick scan of your script and I might have missed it not sure why you are needing to use an analyst to perform OR at very minimum why you are using Trandet record vs. the summary record. T his and should be able to do this with the TERMFIID Associate on the Decline % and DENYCNT for the specific node value "VISA" in where clause.. seems much more efficent.

Doing both of the above would significantly reduce memory utilization.

[TushR]

@ChristopherS ,

Thank you for your response.
Sorry for posting old analyst code, actually I applied where clause on TRANDET record to get particular BIN or FIID transactions.

We can't use any summary record because we are not looking for more declined rate or declined count transactions . Our requirement is to detect 10 (Or any threshold value) CONSECUTIVE declines from particular BIN/FIID irrespective of time interval or decline%. For Eg. For
Scenario 1: BIN 444433 Transaction pattern: A,D,D,D,D,D,D,D,D,D,D,A. Here it should send an alert.
Scenario 2: BIN 444433 Transaction pattern: A,D,D,D,D,D,D,D,D,A,D,D. It should not send an alert as its not 10 CONSECUTIVE declines.

Updated Analysts is as follows :

SECTION CONFIG
  MAX_SIZE 5.0 MB 
  NUMERIC TXNCOUNT [0] := 0
  NUMERIC ALERTFLAG [0] := 1
END_SECTION
 
SECTION RULE_DEF

! Get all records from TRANDET record and set counter 
! increment the counter if it is declined response code and reset the counter if approve response code

RULE TRAN_COUNT PRIMARY

  RECORD TRANDET
  WHERE BIN = "402872" AND (SRCNAME="NSX7DRP2" OR SRCNAME="NSX7DRA2")
  REFRESH 30 SECONDS
  NODE ( #NI\BASE24-active ) 
  SORT DESC LOGTIME 
 
!Increment the counter when the transaction is declined
 
  IF @TRANDET.RESPCODE NOT IN {"00","01","000","001","076","206"}
        SET TXNCOUNT := TXNCOUNT + 1                 
  END_IF
 
!Reset the counter when the transaction is approved
  
  IF @TRANDET.RESPCODE = "00" OR @TRANDET.RESPCODE = "01" OR @TRANDET.RESPCODE = "000" OR @TRANDET.RESPCODE = "001"
        SET TXNCOUNT := 0
  END_IF
 
END_RULE


!Check if the counter value is greater than the threshold and if the send alert flag is on (1)
!If the condition is true, set the alert flag to off (0) to avoid the sending of multiple alerts

RULE SEND_ALERT PRIMARY

  RECORD PNODES
  WHERE ALL
  REFRESH 30 SECONDS
  NODE ( #NI\ConsDecAnalystNode )

  IF TXNCOUNT > 9 AND ALERTFLAG = 1
        LOG 
        SNMPTRAP 0101
          VERSION v1
          HOST          10.129.88.82
          PORT          162
          PRIORITY      CRITICAL
          IMMEDIATELY
        EXEC          #DISPMAN 0100 IMMEDIATELY
        SET ALERTFLAG := 0
        SET TXNCOUNT := 0
  END_IF

END_RULE
 
 
!Rule to reset the send alert flag to on (1) and reset the counter to 0 after x seconds/minutes 
!x seconds/minutes refresh has to match the refresh interval in the rule above
 
RULE RESET_ALERT_FLAG PRIMARY
 
  RECORD PNODES
  WHERE ALL
  REFRESH 30 SECONDS
  NODE ( #NI\BASE24-active ) 
 
  ACTION
        SET ALERTFLAG := 1
        !SET TXNCOUNT := 0
  END_ACTION
 
END_RULE
 
END_SECTION
 
SECTION MSG_TEXT
   MSG 0100  "DISPREQ CMD_SRC 402872ConsDenial MSG_CODE_NUM 100 MSG_DEST ConsDecGroup MSG_TITLE Consecutive BIN 402872 decline @TXNCOUNT@ txns MSG_TEXT Consecutive BIN 402872 decline @TXNCOUNT@ txns."
   MSG 0101  "Consecutive BIN 402872 decline more than 10 transactions"
END_SECTION

[Leon_Arens_FNNI]

Tushar,

    I noticed that have a global variable TXNCOUNT that is incremented by 1 for every decline and reset as soon as an approval is encountered.  

    You also have a rule that will send an alert every 30 seconds if the TRANCOUNT is 10 or more.

    If you have a scenario of A,D,D,D,D,D,D,D,D,D,D,A  in 30 seconds, the result of TRANCOUNT when the the TRAN_COUNT rule completes would be 0.  This would mean that when the SEND_ALERT rule triggers the TRANCOUNT would not be > 9 so it would not trigger.  Are you getting alerts from this analyst?

Christopher,

     You suggested  the the rule be updated to "LOGTIME >  ( CurrentTime - 30 seconds )" to prevent it from collecting more transactions than what happened in the last 30 seconds.  If there are 10 transactions occuring every 30 seconds, how many transaction is the rule processing every 30 seconds if the rule does not have this extra code?

[TushR]

@Leon_Arens_FNNI 

Yes we are receiving alerts for this analysts but I am not sure how analyst will work in your mentioned scenario.

Problem with analysts are that we can't debug it as we usually do in development tools. I don't know how to check whether it trigger an alert for your mentioned scenario.

~tushR

[Leon_Arens_FNNI]

In my experience with Analysts when collecting at 30 second intervals is that they trigger at the Top of the minute and at the bottom of the minute.    i.e. 2:00:00 then 2:00:30 then 2:01:00 then 2:01:30 and so on.

Looking at the SEND_ALERT rule the TXNCOUNT must be > 9 at those times, in order for the rule to trigger.

I am assuming the TRAN_COUNT rule is processing only transactions that occurred in the last 30 seconds.  

If you had scenario of A D D D D D D D D D D A A A A in 30 seconds then the SEND_ALERT would probably not trigger.

If you had a scenario of A D D D D D D D D D D D then the TXNCOUNT would be set to 11 for 30 seconds and the SEND_ALERT would trigger.

Is the SEND_ALERT rule sending alerts on every occurrence of 10 or more consecutive declines?

Are you wanting it to triggering for every occurrence of 10 or more decline transactions in a row, even if they happened in 2 different 30 second collection intervals?

i.e.

between 2:00:00 and 2:00:30  Scenario A A A A A A A A A D D D D D

between 2:00:30 and 2:01:00  Scenario D D D D D A A A A A A A A A 

[ChristopherS]

@Leon_Arens_FNNI 

The reason you want to do this is that without a LIMIT or a timeframe clause I believe the Record clause will look at whatever you have configuration of of IRTSVCOL.ini for the TSV / TRANDET cache which can be anywhere from 1-999,999 transactions; keep in mind though TSV Cache works differently than summary and most other areas of Prognosis.   In a display this is the equivalent of opening this record on a 30 second interval with no limit clause.. it would try to return all rows of the cache and refresh the list every 30 seconds.

---

@Leon_Arens_FNNI wrote:

Christopher,

     You suggested  the the rule be updated to "LOGTIME >  ( CurrentTime - 30 seconds )" to prevent it from collecting more transactions than what happened in the last 30 seconds.  If there are 10 transactions occuring every 30 seconds, how many transaction is the rule processing every 30 seconds if the rule does not have this extra code?

[ChristopherS]

Yes, you cannot 'step' through an analyst with a debugger but you can absolutely debug them through the use of LOG lines which write out the PROBLEM (Not Problem Summary Record).   

In the Analyst training we now do I describe how to do this and have some examples of some best practices to granularly debug everything or individual rules or clauses in a managable way.  Someone else is delivering the content for IRConnect recordings but believe they are using my standard content where I had this detail in there. 

@TushR wrote:

Problem with analysts are that we can't debug it as we usually do in development tools. I don't know how to check whether it trigger an alert for your mentioned scenario.

~tushR

---

[ChristopherS]

@Leon_Arens_FNNI 

Almost everything in Prognosis Server (Analysts, PQL, Thresholds, Displays) are clock aligned and another reason why sticking to 10s, 30s, 60s, etc are important for manging. Analysts do have some unfortunately hard coded delays hard coded into them still when invoking secondary rules (believe its 10 or 20s) and so sometimes to both speedup Analysts or simply sycncronize things you may want to deviate from the standard 10/30/60 rule.. but usually these are done on secondary rules or rules using placeholder records like PVIEWS (where you actually are not using any field in PVIEWs but just checking variables). 

---

@Leon_Arens_FNNI wrote:

In my experience with Analysts when collecting at 30 second intervals is that they trigger at the Top of the minute and at the bottom of the minute.    i.e. 2:00:00 then 2:00:30 then 2:01:00 then 2:01:30 and so on.