Body: <p>I was wondering if there is an algorithm that helps create 'symmetrical' AIs (an AI that has the same inputs as it has outputs) that could be used for 'simulating' a system.</p>
  <p>The idea would be that each output is influenced by every input APART from its counterpart.</p>
  <p>IE. if you have a model with three inputs (IA, IB and IC) and the matching outputs (OA, OB and OC), OA would only be influenced by IB and IC, OB would only be influenced by IA and IC and OC would only be influenced by IA and IB.</p>
  <p>You'd then be able to train your model on history, and end up with something that is a multi-channel regressor. Unlike using a one-class SVM (which you can train to provide a digital output of 'novelty' of a system state), it would be great to get analogue estimate of what each input value <strong>should</strong> be, which is far more useful.</p>
  <p>This would be trivial in a single layer NN, but becomes incredibly complex when there are hidden layers involved. So I doubt an neural network is going to be the answer. I was curious if there was another algorithm that might be more suitable to the problem.</p>
  <p>And yes, I know it's not difficult to model these individually - but I have some relatively large applications, and training huge numbers of individual AIs to model every distinct channel is a little daunting.</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-02-18T12:56:53.480Z
Comments.Text: Are A, B and C the same data type?
Comments.UserDisplayName: Neil Slater
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-02-18T15:08:02.427Z
Comments.Text: Sounds like several questions ("simulating"; "what each input value should be" (according to what criteria?)). Unclear why you mention modeling every distinct channel. Concepts related to simulating: recurrent neural networks, denoising autoencoders, diffusion generative models, trainable simulations, neural ODEs, ... Concepts related to what each input value should be: optimizing over the input (adversarial examples, artistic style transfer using iterations, ...), ... All that doesn't matter because you can simply train neural networks on the history data you mention. Why is that "daunting"?
Comments.UserDisplayName: root
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-02-19T04:08:23.527Z
Comments.Text: @NeilSlater - yes, they are identical. I'd imagine that they'll generally be normalised to floating point numbers even if they start out as integers.
Comments.UserDisplayName: Steve Harding
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-02-19T04:19:28.400Z
Comments.Text: @root It's not really several questions - I was hoping there was a technique that would make for a single answer.
  
  If I had inputs A,B & C I'd like to train an AI so that it can predict the value of A from B & C and at the SAME TIME predict the value of B from A & D. Given that I'll easily have more than 2000 inputs, training 2000 different neural networks is the 'daunting' task.
  
  The only catch is that any sensible AI would simply learn to copy the value directly from A to A, B to B etc. I want to avoid that and ensure that the value of input A has no effect in output A.
Comments.UserDisplayName: Steve Harding
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-02-19T04:44:48.017Z
Comments.Text: @root - 'denoising autoencoders' are trying to deal with a similar issue (the direct mapping of input to output), but I'm not trying to go via an encoded middle-layer.
  
  Because I'm looking for a regressor instead of a generative AI, I found that randomly zeroing-out channels during training caused quite a bit of accuracy loss. 
  
  But I can take another look - perhaps running a bunch of tests with different rates of Dropout layer will come in handy.
Comments.UserDisplayName: Steve Harding
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-02-19T08:01:22.283Z
Comments.Text: A denoising autoencoder won't help you. The problem is more generally stated as leakage, and you cannot afford to have any single route for input data to influence the output of its own prediction. Second question - is there any relationship between the different input/outputs that can be quantified? E.g. are they in a strict sequence, or related by time, or in any other way that might be useful
Comments.UserDisplayName: Neil Slater
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-02-19T12:34:57.157Z
Comments.Text: @SteveHarding 1. Why is that "daunting"? Just write an outer loop. 2. If you refer to keeping the values unchanged as "sensible", why do you try to change them and to hide the previous value from the method? 3. I don't know what you mean by "go via an encoded middle-layer". 4. The rate of dropout or related techniques ("randomly zeroing-out channels") is a hyperparameter that you can tune to slightly improve the results; but other properties of the architecture are probably much more important for getting good first results.
Comments.UserDisplayName: root
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-03-14T08:28:17.120Z
Comments.Text: @root - As for 'daunting', they've all got to run based on live data, so executing hundreds or thousands of individual models every second is a lot more expensive than running a single model once.
  
  "Go by an encoded middle layer" is roughly describing what autoencoders do - having a deliberately very small inner layer to force the output to be more abstract and less directly coupled to the input. 
  
  I found dropout awkward - since I'm making a regressor, and my inputs are varied, it pulled my results down significantly. Might have had dropout too close to the input layer though.
Comments.UserDisplayName: Steve Harding
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-03-14T08:36:57.830Z
Comments.Text: @NeilSlater - no relationships that I can really leverage - certainly nothing linear or consistent, since each input represents a different measurement at different scale. While some inputs may correlate with each-other, that's not usually the case and there's no real 'order' to use convolution on.
  
  I actually have had luck with a suitably tuned autoencoder - a variational autoencoder seems to break any direct reliance from input-to-output unless it has a massive effect on overall output state. Might not be perfect, but it's a LOT closer than I had.
Comments.UserDisplayName: Steve Harding
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-03-14T12:54:24.037Z
Comments.Text: I can't make sense of the question. I think there might be something that could help, but not expressed in the question. Are you able to share what these measurements represent, and why you want to predict inputs you already have, based on all other inputs in same set
Comments.UserDisplayName: Neil Slater
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-03-20T20:50:43.667Z
Comments.Text: Re "expensive": Not likely. What model size have you tried on what hardware?
Comments.UserDisplayName: root
Tags: <algorithm-request>
Title: Symmetrical AI (same inputs as outputs) for Modelling and Prediction
UserDisplayName: Steve Harding
ThreadText: # Symmetrical AI (same inputs as outputs) for Modelling and Prediction
  
  (Asked by: username_0 on 2024-02-18)
  
  I was wondering if there is an algorithm that helps create 'symmetrical' AIs (an AI that has the same inputs as it has outputs) that could be used for 'simulating' a system.
  
  The idea would be that each output is influenced by every input APART from its counterpart.
  
  IE. if you have a model with three inputs (IA, IB and IC) and the matching outputs (OA, OB and OC), OA would only be influenced by IB and IC, OB would only be influenced by IA and IC and OC would only be influenced by IA and IB.
  
  You'd then be able to train your model on history, and end up with something that is a multi-channel regressor. Unlike using a one-class SVM (which you can train to provide a digital output of 'novelty' of a system state), it would be great to get analogue estimate of what each input value **should** be, which is far more useful.
  
  This would be trivial in a single layer NN, but becomes incredibly complex when there are hidden layers involved. So I doubt an neural network is going to be the answer. I was curious if there was another algorithm that might be more suitable to the problem.
  
  And yes, I know it's not difficult to model these individually - but I have some relatively large applications, and training huge numbers of individual AIs to model every distinct channel is a little daunting.
  
  **Tags:** algorithm-request
  
  > **Comments:**
  > **username_1**: Are A, B and C the same data type?  
  > **username_2**: Sounds like several questions ("simulating"; "what each input value should be" (according to what criteria?)). Unclear why you mention modeling every distinct channel. Concepts related to simulating: recurrent neural networks, denoising autoencoders, diffusion generative models, trainable simulations, neural ODEs, ... Concepts related to what each input value should be: optimizing over the input (adversarial examples, artistic style transfer using iterations, ...), ... All that doesn't matter because you can simply train neural networks on the history data you mention. Why is that "daunting"?  
  > **username_0**: @username_1 - yes, they are identical. I'd imagine that they'll generally be normalised to floating point numbers even if they start out as integers.  
  > **username_0**: @username_2 It's not really several questions - I was hoping there was a technique that would make for a single answer.  If I had inputs A,B & C I'd like to train an AI so that it can predict the value of A from B & C and at the SAME TIME predict the value of B from A & D. Given that I'll easily have more than 2000 inputs, training 2000 different neural networks is the 'daunting' task.  The only catch is that any sensible AI would simply learn to copy the value directly from A to A, B to B etc. I want to avoid that and ensure that the value of input A has no effect in output A.  
  > **username_0**: @username_2 - 'denoising autoencoders' are trying to deal with a similar issue (the direct mapping of input to output), but I'm not trying to go via an encoded middle-layer.  Because I'm looking for a regressor instead of a generative AI, I found that randomly zeroing-out channels during training caused quite a bit of accuracy loss.   But I can take another look - perhaps running a bunch of tests with different rates of Dropout layer will come in handy.  
  > **username_1**: A denoising autoencoder won't help you. The problem is more generally stated as leakage, and you cannot afford to have any single route for input data to influence the output of its own prediction. Second question - is there any relationship between the different input/outputs that can be quantified? E.g. are they in a strict sequence, or related by time, or in any other way that might be useful  
  > **username_2**: @username_0 1. Why is that "daunting"? Just write an outer loop. 2. If you refer to keeping the values unchanged as "sensible", why do you try to change them and to hide the previous value from the method? 3. I don't know what you mean by "go via an encoded middle-layer". 4. The rate of dropout or related techniques ("randomly zeroing-out channels") is a hyperparameter that you can tune to slightly improve the results; but other properties of the architecture are probably much more important for getting good first results.  
  > **username_0**: @username_2 - As for 'daunting', they've all got to run based on live data, so executing hundreds or thousands of individual models every second is a lot more expensive than running a single model once.  "Go by an encoded middle layer" is roughly describing what autoencoders do - having a deliberately very small inner layer to force the output to be more abstract and less directly coupled to the input.   I found dropout awkward - since I'm making a regressor, and my inputs are varied, it pulled my results down significantly. Might have had dropout too close to the input layer though.  
  > **username_0**: @username_1 - no relationships that I can really leverage - certainly nothing linear or consistent, since each input represents a different measurement at different scale. While some inputs may correlate with each-other, that's not usually the case and there's no real 'order' to use convolution on.  I actually have had luck with a suitably tuned autoencoder - a variational autoencoder seems to break any direct reliance from input-to-output unless it has a massive effect on overall output state. Might not be perfect, but it's a LOT closer than I had.  
  > **username_1**: I can't make sense of the question. I think there might be something that could help, but not expressed in the question. Are you able to share what these measurements represent, and why you want to predict inputs you already have, based on all other inputs in same set  
  > **username_2**: Re "expensive": Not likely. What model size have you tried on what hardware?  
  
  ---
  
  ## 0 answers
  
  