Body: <p>I am trying to train a CNN in keras to learn a general representation of a Lua module, e.g. requires at the beginning, local variables, local functions, interface (returns) and in between some runnable code (labeled "other"). So for each module (source code) I generate an AST which I then encode in a json file. The file contains the order of the node in the AST, the text it represents and the type of node it is <code>(require, variable, function, interface, other)</code>. It can contain other metrics but so far I have settled on these three, where only the order and type of node will be converted into a vector to serve as input to the CNN. Now I don't have any labels for these modules (I want to treat one module as one input to the CNN), so I have concluded that I need to use unsupervised learning. In keras this should translate to using autoencoders, where I use the encoder part to learn weights for the representation and then connect a fully-connected layer and generate an output. Now before I specify the output, I want to specify the input more closely. In my mind It should be a 3D vector let's say <code>(x,y,z)</code>. <code>x</code> represents the number of nodes of an AST that are taken into consideration, <code>y</code> represents the local neighborhood of said node (for now I have settled on 5 nodes) and <code>z</code> should represent the node itself, the order and type of node. So with that, I would want the output of the network to be in the (almost) same dimension. I want <code>x</code> outputs for every node that was taken as input and a number (ideally between 0-1) to specify how "correct" the node under consideration is in response to the learned representation. My question as a beginner to neural networks is, how feasible is this and are there any points which are simply impossible to do or are wrongly interpreted on my part?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-02-24T13:55:51.687Z
Comments.Text: What are you trying to achieve? I don't quite understand that. When the NN is trained, what will the input and desired output be?
Comments.UserDisplayName: Oliver Mason
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-02-24T14:27:05.173Z
Comments.Text: The input will be a "module" - a file with source code. From this module an AST is created, and nodes of AST are selected as input. For each node a local neighborhood of nodes will be selected (of size 5 nodes). The information a node contains is its position in the AST (1, 2, ..., x) and the node type (require, function, ...). So the input vector for one module will be (x,y,z), with x the number of nodes, y the size of local neighborhood and z the node information. So a 3D vector. The output should be a 1D vector of size X, that say how well placed a node in input module is.
Comments.UserDisplayName: Michael Kročka
Tags: <convolutional-neural-networks><tensorflow><keras><unsupervised-learning>
Title: Training an unsupervised convolutional neural network to learn a general representation of a Lua module
UserDisplayName: Michael Kročka
ThreadText: # Training an unsupervised convolutional neural network to learn a general representation of a Lua module
  
  (Asked by: username_0 on 2020-02-24)
  
  I am trying to train a CNN in keras to learn a general representation of a Lua module, e.g. requires at the beginning, local variables, local functions, interface (returns) and in between some runnable code (labeled "other"). So for each module (source code) I generate an AST which I then encode in a json file. The file contains the order of the node in the AST, the text it represents and the type of node it is `(require, variable, function, interface, other)`. It can contain other metrics but so far I have settled on these three, where only the order and type of node will be converted into a vector to serve as input to the CNN. Now I don't have any labels for these modules (I want to treat one module as one input to the CNN), so I have concluded that I need to use unsupervised learning. In keras this should translate to using autoencoders, where I use the encoder part to learn weights for the representation and then connect a fully-connected layer and generate an output. Now before I specify the output, I want to specify the input more closely. In my mind It should be a 3D vector let's say `(x,y,z)`. `x` represents the number of nodes of an AST that are taken into consideration, `y` represents the local neighborhood of said node (for now I have settled on 5 nodes) and `z` should represent the node itself, the order and type of node. So with that, I would want the output of the network to be in the (almost) same dimension. I want `x` outputs for every node that was taken as input and a number (ideally between 0-1) to specify how "correct" the node under consideration is in response to the learned representation. My question as a beginner to neural networks is, how feasible is this and are there any points which are simply impossible to do or are wrongly interpreted on my part?
  
  **Tags:** convolutional-neural-networks, tensorflow, keras, unsupervised-learning
  > **Comments:**
  > **username_1**: What are you trying to achieve? I don't quite understand that. When the NN is trained, what will the input and desired output be?  
  > **username_0**: The input will be a "module" - a file with source code. From this module an AST is created, and nodes of AST are selected as input. For each node a local neighborhood of nodes will be selected (of size 5 nodes). The information a node contains is its position in the AST (1, 2, ..., x) and the node type (require, function, ...). So the input vector for one module will be (x,y,z), with x the number of nodes, y the size of local neighborhood and z the node information. So a 3D vector. The output should be a 1D vector of size X, that say how well placed a node in input module is.  
  
  ---
  
  ## 0 answers
  
  