I will admit to not following AI at all for about 20 years, so perhaps this is old hat now, but having separate policy networks and value networks is quite ingenious.  I wonder how successful this would be at natural language generation.  It reminds me of Krashen&#x27;s theories of language acquisition where there is a &quot;monitor&quot; that gives you fuzzy matches on whether your sentences are correct or not.  One of these days I&#x27;ll have to read their paper.