Answers.Body: <p>I think you would find this link helpful. It demonstrates how to identify patterns in large arbitrary byte data.</p>
  
  <p><a href="https://devblogs.nvidia.com/malware-detection-neural-networks/" rel="nofollow noreferrer">https://devblogs.nvidia.com/malware-detection-neural-networks/</a></p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-08-08T20:36:18.667Z
Answers.LastActivityDate: 2018-08-08T20:36:18.667Z
Answers.UserDisplayName: Zakk Diaz
Answers.Body: <p>You are trying to solve a variant of <a href="https://en.wikipedia.org/wiki/Halting_problem" rel="nofollow noreferrer">The Halting Problem</a>, which is the problem of detecting whether a computer program is going to stop, or run forever.</p>
  
  <p>The Halting Problem is incomputable, which means it is not possible to write a computer program that solves it. It is easy to see that your problem is also incomputable. If you could predict whether a program would generate an error, then for any program X that someone wants to solve the halting problem for, we could write a new program:</p>
  
  <ol>
  <li>run(X)</li>
  <li>Error("X finished running").</li>
  </ol>
  
  <p>and use your algorithm to determine whether X would finish running or not. Since the halting problem is known to be incomputable, this means your problem must be incomputable too.</p>
  
  <p>That's not to say all is lost though. Formal verification is a field that uses some AI techniques (mostly reasoning-based, but I think there's some machine learning now too) to try to solve this problem for some programs. It can't work for <em>every</em> program though.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-08-08T16:40:04.100Z
Answers.LastActivityDate: 2018-08-08T16:40:04.100Z
Answers.UserDisplayName: John Doucette
Answers.Body: <p>Debug and Validation of large software systems like video software stack
  If you take example as validation and debug of video software stack, It is very difficult for naked eye to identify failures on the display. In this case you can use DNN based image classifier to identify functional failure.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-09-17T18:05:10.213Z
Answers.LastActivityDate: 2018-09-17T18:05:10.213Z
Answers.UserDisplayName: Sreedhar Veluri
Answers.Body: <p>If you consider also 3rd party options instead of building your own system, you may want to check this solution <a href="https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch_Anomaly_Detection.html" rel="nofollow noreferrer">https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch_Anomaly_Detection.html</a></p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-08-26T07:39:38.417Z
Answers.LastActivityDate: 2023-08-26T07:39:38.417Z
Answers.UserDisplayName: Putnik
Body: <p>I'm trying to develop a kind of AI that will assist in debugging a large software system while we run test cases on it. The AI can get access to anything that the developers of the system can, namely log files, and execution data from trace points. It also has access to the structure of the system, and all of the source code.</p>
  
  <p>The end goal of this AI is to be able to detect runtime errors during execution, and locate the source of these errors.</p>
  
  <p>I was considering making use of a deep neural network, where the input would be the execution data and log output. Using this input it would be able to verify whether the current version of the system we are running is functional, or non-functional. The problems with this approach is that the system it would be evaluating would be constantly changing as it gets developed, so the only training material the NN would have is from the last stable version of the system (and even that could have some errors). Additionally, producing test cases for the system off of which we could train the NN would be very time consuming, and would defeat the purpose of using the NN in the first place.</p>
  
  <p>I would like to know what AI design you think would be suitable for this task. Please let me know if you would like any other information relevant to the problem. As far as I can tell, nothing quite like this has been done before.</p>
  
  <p>It's probably worth mentioning that my team has a some extremely powerful machines on which we can run the AI.</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-06-05T16:44:49.187Z
Comments.Text: Is the programming language you are using Turing Complete?
Comments.UserDisplayName: MilkyWay90
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-05-11T10:30:03.437Z
Comments.Text: I dunno neural networks very well. For what you are trying to achieve, I guess you need to first freeze the state and do the execution. If your task is telling if the system is functional or non functional, it can be thought of as a classification task. You can start off looking at using DNN for classification. I know this is a broad overview, however, should give you some lead.
Comments.UserDisplayName: okkhoy
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-05-11T11:06:37.930Z
Comments.Text: Could you please try to re-phrase your question.ideally,we don't have types of AIs but specific fields where your concept idea could fall given the problem is solved  ie machine learning,In this case a piece program once coded effectively, can learn from previous errors or data logs and keeps on advancing thus helping you at the end;debugging your code.You need to be kind specific.However,CybergrandChallenge one team did the same program and i think it's being worked on there! Ain't asleep!!.
Comments.UserDisplayName: quintumnia
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-05-11T15:31:32.460Z
Comments.Text: Let me comment on **assist in debugging a large software system** : I've worked on a subclass of that problem with great success using linear regression on code features.  Rephrasing the question, or changing the requirements slightly could make it a lot easier.  For example, if your specific objective was **assign a probability to each function that a defect will be discovered in our next round of regression tests** and you had data of which functions had defects from the last regression, then that is very doable!
Comments.UserDisplayName: Anton Panchishin
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2018-09-17T18:38:48.583Z
Comments.Text: you are not clear on what is it exactly that you want,
  you want an AI that can run your program for different inputs and log errors if they exists?
Comments.UserDisplayName: Moher
Tags: <machine-learning><deep-learning><ai-design>
Title: What type of AI would you recommend for this complex problem?
UserDisplayName: Will Da Silva
ThreadText: # What type of AI would you recommend for this complex problem?
  
  (Asked by: username_0 on 2017-05-09)
  
  I'm trying to develop a kind of AI that will assist in debugging a large software system while we run test cases on it. The AI can get access to anything that the developers of the system can, namely log files, and execution data from trace points. It also has access to the structure of the system, and all of the source code.
  
  The end goal of this AI is to be able to detect runtime errors during execution, and locate the source of these errors.
  
  I was considering making use of a deep neural network, where the input would be the execution data and log output. Using this input it would be able to verify whether the current version of the system we are running is functional, or non-functional. The problems with this approach is that the system it would be evaluating would be constantly changing as it gets developed, so the only training material the NN would have is from the last stable version of the system (and even that could have some errors). Additionally, producing test cases for the system off of which we could train the NN would be very time consuming, and would defeat the purpose of using the NN in the first place.
  
  I would like to know what AI design you think would be suitable for this task. Please let me know if you would like any other information relevant to the problem. As far as I can tell, nothing quite like this has been done before.
  
  It's probably worth mentioning that my team has a some extremely powerful machines on which we can run the AI.
  
  **Tags:** machine-learning, deep-learning, ai-design
  > **Comments:**
  > **username_2**: I dunno neural networks very well. For what you are trying to achieve, I guess you need to first freeze the state and do the execution. If your task is telling if the system is functional or non functional, it can be thought of as a classification task. You can start off looking at using DNN for classification. I know this is a broad overview, however, should give you some lead.  
  > **username_3**: Could you please try to re-phrase your question.ideally,we don't have types of AIs but specific fields where your concept idea could fall given the problem is solved  ie machine learning,In this case a piece program once coded effectively, can learn from previous errors or data logs and keeps on advancing thus helping you at the end;debugging your code.You need to be kind specific.However,CybergrandChallenge one team did the same program and i think it's being worked on there! Ain't asleep!!.  
  > **username_4**: Let me comment on **assist in debugging a large software system** : I've worked on a subclass of that problem with great success using linear regression on code features.  Rephrasing the question, or changing the requirements slightly could make it a lot easier.  For example, if your specific objective was **assign a probability to each function that a defect will be discovered in our next round of regression tests** and you had data of which functions had defects from the last regression, then that is very doable!  
  > **username_5**: you are not clear on what is it exactly that you want, you want an AI that can run your program for different inputs and log errors if they exists?  
  > **username_1**: Is the programming language you are using Turing Complete?  
  
  ---
  
  ## 4 answers
  
  ### Answer by username_6 on 2018-08-08. Score: 1 | Accepted answer
  
  I think you would find this link helpful. It demonstrates how to identify patterns in large arbitrary byte data.
  
  <https://devblogs.nvidia.com/malware-detection-neural-networks/>
  
  
  ---
  
  ### Answer by username_7 on 2018-08-08. Score: 2
  
  You are trying to solve a variant of [The Halting Problem](https://en.wikipedia.org/wiki/Halting_problem), which is the problem of detecting whether a computer program is going to stop, or run forever.
  
  The Halting Problem is incomputable, which means it is not possible to write a computer program that solves it. It is easy to see that your problem is also incomputable. If you could predict whether a program would generate an error, then for any program X that someone wants to solve the halting problem for, we could write a new program:
  
  1. run(X)
  2. Error("X finished running").
  
  and use your algorithm to determine whether X would finish running or not. Since the halting problem is known to be incomputable, this means your problem must be incomputable too.
  
  That's not to say all is lost though. Formal verification is a field that uses some AI techniques (mostly reasoning-based, but I think there's some machine learning now too) to try to solve this problem for some programs. It can't work for *every* program though.
  
  
  ---
  
  ### Answer by username_8 on 2018-09-17. Score: 0
  
  Debug and Validation of large software systems like video software stack
  If you take example as validation and debug of video software stack, It is very difficult for naked eye to identify failures on the display. In this case you can use DNN based image classifier to identify functional failure.
  
  
  ---
  
  ### Answer by username_9 on 2023-08-26. Score: 0
  
  If you consider also 3rd party options instead of building your own system, you may want to check this solution <https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch_Anomaly_Detection.html>
  
  
  ---
  
  