Answers: <p>It depends on what you mean by "log files" I supposed.  If you mean the telemetry data you use to verify your system is operating correctly I would say "Don't log sensitive fields."  You don't need that kind of information to alert you to high or low transaction rates, response times to your dependent services, etc.</p>
  
  <p>If you mean data for billing or audit purposes I would suggest you establish a write-only pipeline where the data is written but can't be read by the writer.  Then your billing and auditing  pipeline kicks in and you have the controls there to audit who looked at individual records.</p>
  
  <p>In the end security comes down to documented processes with their own secure log files, etc.  At some point you have to trust someone or some process since all data is written so that it can later be read.</p>
  
Answers: CC BY-SA 3.0
Answers: 2017-03-27T21:03:12.500Z
Answers: down vote with no comment.  nice.  Trolls everywhere.
Answers: No Refunds No Returns
Answers: CC BY-SA 3.0
Answers: 2017-03-26T21:27:44.327Z
Answers: 2017-03-26T21:27:44.327Z
Answers: No Refunds No Returns
Answers: <p>I think the solution comes down to a broad spectrum of approaches that ensures data protection:</p>
  
  <ul>
  <li><p><strong>Data Classification</strong>: The most efficient technical strategy is to categorise the data at the point of creation rigorously. At its core, the developers are responsible for ensuring that all logged information is assigned a category.  Categorization can, for example, be achieved through <a href="https://docs.splunk.com/Documentation/Splunk/6.5.2/SearchReference/Metadata" rel="nofollow noreferrer">Splunk Metadata</a> which, in turn, can be used to direct log entries to different buckets based upon their data categorization.</p></li>
  <li><p><strong>Event Partitioning</strong>: There is often a desire to log sensitive information along side non-sensitive information. For example, if a new user was to sign up you may log:</p>
  
  <ul>
  <li>Customer ID (Restricted)</li>
  <li>Customer Type (Restricted)</li>
  <li>Aquisition Source (Unrestricted)</li>
  <li>Correlation ID (Unrestricted)</li>
  </ul>
  
  <p>It is possible to split this one "Event" into two parts, one containing the <em>Restricted</em> information and one containing th <em>Unrestricted</em> information.  This aligns with the first point by allowing filtering rules to direct to different buckets.</p></li>
  <li><p><strong>Data Masking</strong>: In specific circumstances it may not be possible to categorise the data at source, in my experience logging solutions do allow for <a href="https://answers.splunk.com/answers/211998/masking-sensitive-data.html" rel="nofollow noreferrer">Masking Rules</a> to X out sensitive data.  In the linked example a <code>sed</code> command is used to apply a regular expression to all data from a specific source.  Once the restricted data has been masked out then the event can be considered to be Unrestricted.  Care must be taken with the rule to ensure that information critical to an event such as a correlation ID doesn't match the Regular Expression used to match sensitive data.</p></li>
  <li><p><strong>Event Filtering</strong>: As a last resort it may be necessary to filter all events of a particular type or source into a separate bucket if they contain sensitive data that cannot be categorised or masked.  In this case the information in a restricted bucket could only be accessed through a <a href="https://security.stackexchange.com/questions/14430/in-healthcare-it-is-a-break-glass-mode-to-bypass-access-controls-commonly-req">Break-Glass mechanism</a> to bypass access controls under the provisions of an incident.</p></li>
  </ul>
  
  <p>With each of these solutions, testing is key to ensuring that nothing slips through the cracks.  Logging and Event Management must be considered to be a first class Non-functional Requirement with the same level of development rigour applied to a solution - including peer review and tests to ensure that data is properly categorised and partitioned in the tool of choice. </p>
  
Answers: CC BY-SA 3.0
Answers: 2017-03-21T08:43:51.137Z
Answers: 2017-03-21T08:43:51.137Z
Answers: Richard Slater
Body: <p>Working in highly regulated environments data is classified in different ways depending on the sensitivity. In some cases, this is legally enforced and must be treated differently.</p>
  
  <p>Examples of a data classification policy are:</p>
  
  <ul>
  <li><strong>Highly Restricted</strong> Data, such as Passwords, Private Keys, SAML Tokens, and Credit Card Numbers.</li>
  <li><strong>Restricted</strong> Data, such as Usernames and Customer IDs.</li>
  <li><strong>Unrestricted</strong> Data, pretty much anything else.</li>
  </ul>
  
  <p>This classification comes with certain obligations:</p>
  
  <ul>
  <li><p>Any data that is <strong>Highly Restricted</strong> must never be made available in log file under any circumstances.</p></li>
  <li><p><strong>Restricted</strong> data could be made available in log files under specific conditions. For example, if an incident were to occur with a service then the on-call engineer would be able to enact a <a href="http://security-architect.com/how-to-balance-assurance-and-availability-in-pam-systems/" rel="nofollow noreferrer">Break-Glass</a> procedure to access this data to diagnose the problem. The Break-Glass procedure would, in turn, trigger a review, an audit, and possibly temporary revocation of privilege from that engineer.</p></li>
  </ul>
  
  <p>What strategies can be employed to achieve this, particularly given that there are a broad range of logging, monitoring and instrumentation tools available on the market that don't provide a direct answer to this problem?</p>
  
  <p>For example, both Splunk and AppDynamics have the ability to impose different access controls upon conditions of the telemetry being exposed; this means you can create a filter that masks out <code>&lt;customerId&gt;NNNNNNNNNNNN&lt;/customerId&gt;</code>.  However, neither of these tools give you the capability automatically identify credit card numbers and mask them automatically.</p>
  
  <p><strong>Note</strong>: I believe this is related to DevOps because in a traditional tiered support model a relatively small group of people could have access to the data and filter it manually, by devolving responsibility for operating platforms to development teams this data is potentially exposed to a far wider audience.</p>
  
Comments: CC BY-SA 3.0
Comments: 2017-03-18T13:23:25.923Z
Comments: Is the question about closed source / Saas system, or more generic to include ELK stack also ?
Comments: Tensibai
Comments: CC BY-SA 3.0
Comments: 2017-03-18T14:00:31.153Z
Comments: I suspect anything that applies to the SaaS logging and telemetry products could be applied to Open Source tools.  There may be features that support this, like Splunks ability to route to different buckets based upon rules or data masking but those are also available on Open Source Stacks to one extent and another.
Comments: Richard Slater
Comments: CC BY-SA 3.0
Comments: 2017-03-18T14:10:01.350Z
Comments: @Pierre.Vriens ELK = ElasticSearch, LogStash and Kibana : https://www.elastic.co/webinars/introduction-elk-stack
Comments: Richard Slater
Comments: CC BY-SA 3.0
Comments: 2017-03-18T14:12:46.067Z
Comments: @RichardSlater merci for the teaching!
Comments: Pierre.Vriens
Comments: CC BY-SA 3.0
Comments: 2017-03-18T14:31:01.177Z
Comments: For the reason between closed source/sans and open source: the same as between on cloud and on premise, the confidence in a third party is a major difference which could be avoided in answer to focus on the technical part if open source on premise products are included, for nos there is only proprietary or saas product you can't audit.... this change how to answer the question
Comments: Tensibai
Comments: CC BY-SA 3.0
Comments: 2017-03-18T15:03:33.147Z
Comments: Thanks for the feedback @Pierre.Vriens and Tensibai I have re-written the last paragraph to focus on the "Strategies" rather than the specific implementations, which was what I was getting at.
Comments: Richard Slater
Comments: CC BY-SA 3.0
Comments: 2017-03-18T15:18:38.020Z
Comments: This question makes me think about "PCI compliance" (about Processing Creditcards Information), which is not just about logfilles, but any data about Creditcards. But still I'm unsure about the DevOps part of this question (sorry).
Comments: Pierre.Vriens
Comments: CC BY-SA 3.0
Comments: 2017-03-18T15:23:45.643Z
Comments: @Pierre.Vriens I belive it is DevOps because whereas before a relatively small group of people could have access to the data and filter it manually, by devolving responsibility for operating platforms to development teams this data is potentially exposed to a far wider audience.
Comments: Richard Slater
Comments: CC BY-SA 3.0
Comments: 2017-03-18T15:30:22.963Z
Comments: Far better focused now :)
Comments: Tensibai
Comments: CC BY-SA 3.0
Comments: 2017-03-18T16:43:29.090Z
Comments: With your last comment I get it (sorry it took so long). Please review the way I tried to include it in your actual question (the Note at the end). You might want to reword/finetune it to fit your own wording (I did as good as I could think of). Let the answers follow!
Comments: Pierre.Vriens
Tags: <security><logging><data><financial-services><data-protection>
Title: What strategies can be employed to secure sensitive data in log files?
UserDisplayName: Richard Slater
ThreadText: # What strategies can be employed to secure sensitive data in log files?
  
  (Asked by: username_0 on 2017-03-18)
  
  Working in highly regulated environments data is classified in different ways depending on the sensitivity. In some cases, this is legally enforced and must be treated differently.
  
  Examples of a data classification policy are:
  
  * **Highly Restricted** Data, such as Passwords, Private Keys, SAML Tokens, and Credit Card Numbers.
  * **Restricted** Data, such as Usernames and Customer IDs.
  * **Unrestricted** Data, pretty much anything else.
  
  This classification comes with certain obligations:
  
  * Any data that is **Highly Restricted** must never be made available in log file under any circumstances.
  * **Restricted** data could be made available in log files under specific conditions. For example, if an incident were to occur with a service then the on-call engineer would be able to enact a [Break-Glass](http://security-architect.com/how-to-balance-assurance-and-availability-in-pam-systems/) procedure to access this data to diagnose the problem. The Break-Glass procedure would, in turn, trigger a review, an audit, and possibly temporary revocation of privilege from that engineer.
  
  What strategies can be employed to achieve this, particularly given that there are a broad range of logging, monitoring and instrumentation tools available on the market that don't provide a direct answer to this problem?
  
  For example, both Splunk and AppDynamics have the ability to impose different access controls upon conditions of the telemetry being exposed; this means you can create a filter that masks out `<customerId>NNNNNNNNNNNN</customerId>`. However, neither of these tools give you the capability automatically identify credit card numbers and mask them automatically.
  
  **Note**: I believe this is related to DevOps because in a traditional tiered support model a relatively small group of people could have access to the data and filter it manually, by devolving responsibility for operating platforms to development teams this data is potentially exposed to a far wider audience.
  
  **Tags:** security, logging, data, financial-services, data-protection
  
  > **Comments:**
  > **username_1**: Is the question about closed source / Saas system, or more generic to include ELK stack also ?  
  > **username_0**: I suspect anything that applies to the SaaS logging and telemetry products could be applied to Open Source tools.  There may be features that support this, like Splunks ability to route to different buckets based upon rules or data masking but those are also available on Open Source Stacks to one extent and another.  
  > **username_0**: @username_2 ELK = ElasticSearch, LogStash and Kibana : https://www.elastic.co/webinars/introduction-elk-stack  
  > **username_2**: @username_0 merci for the teaching!  
  > **username_1**: For the reason between closed source/sans and open source: the same as between on cloud and on premise, the confidence in a third party is a major difference which could be avoided in answer to focus on the technical part if open source on premise products are included, for nos there is only proprietary or saas product you can't audit.... this change how to answer the question  
  > **username_0**: Thanks for the feedback @username_2 and username_1 I have re-written the last paragraph to focus on the "Strategies" rather than the specific implementations, which was what I was getting at.  
  > **username_2**: This question makes me think about "PCI compliance" (about Processing Creditcards Information), which is not just about logfilles, but any data about Creditcards. But still I'm unsure about the DevOps part of this question (sorry).  
  > **username_0**: @username_2 I belive it is DevOps because whereas before a relatively small group of people could have access to the data and filter it manually, by devolving responsibility for operating platforms to development teams this data is potentially exposed to a far wider audience.  
  > **username_1**: Far better focused now :)  
  > **username_2**: With your last comment I get it (sorry it took so long). Please review the way I tried to include it in your actual question (the Note at the end). You might want to reword/finetune it to fit your own wording (I did as good as I could think of). Let the answers follow!  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_0 on 2017-03-21. Score: 4 | Accepted answer
  
  I think the solution comes down to a broad spectrum of approaches that ensures data protection:
  
  * **Data Classification**: The most efficient technical strategy is to categorise the data at the point of creation rigorously. At its core, the developers are responsible for ensuring that all logged information is assigned a category. Categorization can, for example, be achieved through [Splunk Metadata](https://docs.splunk.com/Documentation/Splunk/6.5.2/SearchReference/Metadata) which, in turn, can be used to direct log entries to different buckets based upon their data categorization.
  * **Event Partitioning**: There is often a desire to log sensitive information along side non-sensitive information. For example, if a new user was to sign up you may log:
  
    + Customer ID (Restricted)
    + Customer Type (Restricted)
    + Aquisition Source (Unrestricted)
    + Correlation ID (Unrestricted)
  
    It is possible to split this one "Event" into two parts, one containing the *Restricted* information and one containing th *Unrestricted* information. This aligns with the first point by allowing filtering rules to direct to different buckets.
  * **Data Masking**: In specific circumstances it may not be possible to categorise the data at source, in my experience logging solutions do allow for [Masking Rules](https://answers.splunk.com/answers/211998/masking-sensitive-data.html) to X out sensitive data. In the linked example a `sed` command is used to apply a regular expression to all data from a specific source. Once the restricted data has been masked out then the event can be considered to be Unrestricted. Care must be taken with the rule to ensure that information critical to an event such as a correlation ID doesn't match the Regular Expression used to match sensitive data.
  * **Event Filtering**: As a last resort it may be necessary to filter all events of a particular type or source into a separate bucket if they contain sensitive data that cannot be categorised or masked. In this case the information in a restricted bucket could only be accessed through a [Break-Glass mechanism](https://security.stackexchange.com/questions/14430/in-healthcare-it-is-a-break-glass-mode-to-bypass-access-controls-commonly-req) to bypass access controls under the provisions of an incident.
  
  With each of these solutions, testing is key to ensuring that nothing slips through the cracks. Logging and Event Management must be considered to be a first class Non-functional Requirement with the same level of development rigour applied to a solution - including peer review and tests to ensure that data is properly categorised and partitioned in the tool of choice.
  
  
  ---
  
  ### Answer by username_3 on 2017-03-26. Score: 0
  
  It depends on what you mean by "log files" I supposed. If you mean the telemetry data you use to verify your system is operating correctly I would say "Don't log sensitive fields." You don't need that kind of information to alert you to high or low transaction rates, response times to your dependent services, etc.
  
  If you mean data for billing or audit purposes I would suggest you establish a write-only pipeline where the data is written but can't be read by the writer. Then your billing and auditing pipeline kicks in and you have the controls there to audit who looked at individual records.
  
  In the end security comes down to documented processes with their own secure log files, etc. At some point you have to trust someone or some process since all data is written so that it can later be read.
  
  > **Comments:**
  > **username_3**: down vote with no comment.  nice.  Trolls everywhere.  
  
  ---
  
  