Prognosis Database: Delete data from Database.

[CarstenHegemann]

Hi all,

the problem here is the DSGVO (Germany) on European level it is:

REGULATION (EU) 2016/679 OF THE EUROPEAN PARLIAMENT AND OF THE COUNCIL of 27 April 2016

on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and repealing Directive 95/46/EC (General Data Protection Regulation)

The Data Protection pointed out the problem to delete special Data on our Prognosis Servers.

In our special case we must delete CDR (Call Detail Records) from our Prognosis Databases.

One way is to delete the complete Database with all Data and rebuild it. 

Within Data View Definition we have several Records (Callmanager Call is only one of them) configured in DB.

We would like to keep the main part/most Records of the DB and the collected data only the part/Record which contain

the personal data must be deleted.

Is there a way to delete special data of a Prognosis Database?

I already tested to modify the DB. 

- Start without the special Record and start again the original DB. --- but this did not delete old Data, after this the original Data is still available. (Data was not deleted)

I checked also the Files stored on the Prognosis Server, but I only see cyclical data no Files (.KEY, .DAT, .CNF) that are shared by Records, so this is also not an option.

Maybe someone has an idea!

Carsten

[ChristopherS]

Hi Carsten, I am not aware of anything that can go back and do the equivalent of a drop table for a specific database record from a Prognosis database nor a command to go back and flush those contents. As you already also found out when you change the configuration of a database it stops storing it but doesn't roll off the old information until it reaches the end of the overall storage threshold.

What I am aware you can do is (some of which you have already figured out)

1) Change the configuration of the database and purge all old data and store the specific record(s) which have a different retention schedule into a new database (though if this is an out of the box database that will be complicated).

2) Change the configuration to no longer store the record and wait until the data rolls off.

3) #2 temporarily decrease the retention of the database; though this would purge off all old data not just the Call Detail Records (CDRs).

4) Potentially modify the view of the record to have an additional where clause to limit the timeframe which is accessible. This would only alter the WebUI 'view' of the data but it would limit its viewing.

5) If the detail were in Postgres, the easy way we would be able to address this is go directly to Postgres and maintain a retention there.. but Prognosis databases are not there yet. 

I also did look at PQL and the delete command doesn't apply to Prognosis Databases. I know for GDPR in 11.5 (later in 2018) we are added a Personally Identifiable Information (PII) flag to fields in the UDEFSREC but currently don't have any functions that would allow us to do anything extra to fields with this information like the CUSTOMER option does today.

None of these options are great though and specifically accomplish what you are requesting in a clean way.  I would suggest you file this issue as a potential GDPR compliance issue.

Christopher

[Adam_Annen · Accepted solution]

Hi Carston,

There is no way to run SQL style queries to delete rows in a Prognosis format database. However, I can think of two useful approaches for you.

1. Summarize the data to a new database and apply a where clause at summary time that excludes the particular data you wish to strip.
Online Help topic: http://help.prognosis.com/dr/Prognosis_11-4/mergedProjects/System_Functions/HTML_Files/Database_Summ...
or

2. If the data is only in the later rows (Like the last 5 days) you could adjust the wrap size of the database at that cutoff point and the data will wrap off.

In later versions of prognosis, some data is stored in Mongo or Postgres (Not for everything), and those do respond to clever SQL's to drop rows etc.

I hope you find the above useful. If yes, please do use the 'Accept Solution' button in this topic to let others know this was helpful.

Regards

Adam

[CarstenHegemann]

Hello,

I tested to delete a record by building a DB summary, stop original DB and copy all data from summary folder to database collection folder and start original Database again.

And it works. I think it´s also no problem to use a filter like described.

Thank you