<p>There is this <a href="https://www.researchgate.net/publication/337232925_On_the_Evaluation_Measures_for_Machine_Learning_Algorithms_for_Safety-Critical_Systems" rel="nofollow noreferrer">paper</a> talking specifically about the safety of ML/DL
Algorithm but in industrial application. Since the algorithm is purely learning function you have to define the safety in terms of the application you are using it for example diagnosis or surgery.</p>

CC BY-SA 4.0
2021-09-22T13:23:20.693Z
2021-09-22T13:23:20.693Z
muralee_xo
<p>I think you should first start with definition of software safety in health domain. For example, you should start with <a href="https://en.wikipedia.org/wiki/Therac-25" rel="nofollow noreferrer">Therac-25</a> accident.
Then look at the current scientific articles and standards about software safety in medical domain. Then think about how your algorithm will be tested.</p>
<p>You are thinking Deep RL algorithms as a blackbox but they are software in the end.
If Deep RL algorithms will be used in hospitals, they will have to be tested. The benchmarks, conditions and restrictions of normal software must apply to RL algorithms too.</p>

CC BY-SA 4.0
2021-09-22T13:57:53.577Z
You are correct, in fact, I have been looking for a (healthcare) task to contextualize the safety definition. It's not that I didn't find, there are lots, but the probleme is, these tasks are not available/accessible for me to measure. For example, if I chose the task of sepsis treatment, How Can I measure the safety-related properties of the algorithms. There is no simulation environment or dataset available. This what makes RL widely applicable and measures only on games (given the available benchmarks such as OpenAI gym environments)
mac179
CC BY-SA 4.0
2021-09-22T16:04:27.323Z
Then you need to start with simulator
Atilla Ozgur
CC BY-SA 4.0
2021-09-22T11:32:24.410Z
2021-09-22T11:32:24.410Z
Atilla Ozgur
<p>I applied for a Ph.D. in AI, my advisor told me that my thesis is about safe applications of deep RL algorithms in healthcare. So I decided to do as the first paper, a comparison of Deep RL algorithms in terms of their inherent safety. However, after lots of research, I could not find an answer to my question, that is: How to measure Deep RL algorithms in terms of safety?</p>

<reinforcement-learning><ai-safety>
How to measure Deep RL algorithms in terms of safety?
mac179
# How to measure Deep RL algorithms in terms of safety?

(Asked by: username_0 on 2021-09-22)

I applied for a Ph.D. in AI, my advisor told me that my thesis is about safe applications of deep RL algorithms in healthcare. So I decided to do as the first paper, a comparison of Deep RL algorithms in terms of their inherent safety. However, after lots of research, I could not find an answer to my question, that is: How to measure Deep RL algorithms in terms of safety?

**Tags:** reinforcement-learning, ai-safety


---

## 2 answers

### Answer by username_1 on 2021-09-22. Score: 1

There is this [paper](https://www.researchgate.net/publication/337232925_On_the_Evaluation_Measures_for_Machine_Learning_Algorithms_for_Safety-Critical_Systems) talking specifically about the safety of ML/DL
Algorithm but in industrial application. Since the algorithm is purely learning function you have to define the safety in terms of the application you are using it for example diagnosis or surgery.


---

### Answer by username_2 on 2021-09-22. Score: 1

I think you should first start with definition of software safety in health domain. For example, you should start with [Therac-25](https://en.wikipedia.org/wiki/Therac-25) accident.
Then look at the current scientific articles and standards about software safety in medical domain. Then think about how your algorithm will be tested.

You are thinking Deep RL algorithms as a blackbox but they are software in the end.
If Deep RL algorithms will be used in hospitals, they will have to be tested. The benchmarks, conditions and restrictions of normal software must apply to RL algorithms too.

> **Comments:**
> **username_0**: You are correct, in fact, I have been looking for a (healthcare) task to contextualize the safety definition. It's not that I didn't find, there are lots, but the probleme is, these tasks are not available/accessible for me to measure. For example, if I chose the task of sepsis treatment, How Can I measure the safety-related properties of the algorithms. There is no simulation environment or dataset available. This what makes RL widely applicable and measures only on games (given the available benchmarks such as OpenAI gym environments)  
> **username_2**: Then you need to start with simulator  

---

