Answers.Body: <p>We've been using <a href="http://web-cat.org" rel="nofollow noreferrer">Web-CAT</a>.  It's ok.  It certainly motivates students to read error messages more closely.</p>
  
  <p>EDIT: Here are three things that have bugged me about Web-CAT:</p>
  
  <ul>
  <li>I remember having difficulty determining exactly where my unit tests were supposed to be uploaded; there's a user folder and an assignment folder, and one something's in the wrong place it's hard to just move it.</li>
  <li>In three years, I still haven't found how to get it to accept "Type arrayname[]" as opposed to "Type[] arrayname"; students hate losing those two points.</li>
  <li>Students sometimes have to click around a little to find report details, and the UI doesn't always make clear what's clickable.</li>
  </ul>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-06-30T01:31:21.763Z
Answers.Text: Hi Chris!  This is something of a stub answer.  Would you mind editing it?  Don't worry, the norms of answering on SE sites are not always obvious, and you're definitely not the first one to have a problem like this :) What can you tell us about Web-CAT?  What are its strong points and its weaker points?
Answers.UserDisplayName: Ben I.
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-06-30T12:54:15.450Z
Answers.Text: Thanks for the note.  I'll get the hang of this.  :)  1) I remember having difficulty determining exactly where my unit tests were supposed to be uploaded; there's a user folder and an assignment folder, and one something's in the wrong place it's hard to just move it.  2) In three years, I still haven't found how to get it to accept "Type arrayname[]" as opposed to "Type[] arrayname"; students hate losing those two points.  3) Students sometimes have to click around a little to find report details, and the UI doesn't always make clear what's clickable.
Answers.UserDisplayName: Chris M.
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-06-30T12:55:40.613Z
Answers.Text: Oh, I meant in the answer itself :) Comments are considered temporary content, even if in practice they rarely get deleted.
Answers.UserDisplayName: Ben I.
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-06-30T13:05:10.350Z
Answers.Text: I only just now realized I can edit my answer!  Thanks!
Answers.UserDisplayName: Chris M.
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-06-29T23:56:54.007Z
Answers.LastActivityDate: 2017-06-30T13:08:59.140Z
Answers.LastEditDate: 2017-06-30T13:08:59.140Z
Answers.UserDisplayName: Chris M.
Answers.Body: <blockquote>
    <p>Note: I'm one of the founders of CodeHS.</p>
  </blockquote>
  
  <p>If you check out CodeHS, we have a system for writing autograders using Java. You can write unit test specific problems that test the output of a specific method or write generic autograders for any Java program if you want to test more complex things.</p>
  
  <p>You can read more about them here: <a href="https://medium.com/read-write-code/these-are-the-autograders-you-ve-been-looking-for-bda0fa8fd8a" rel="nofollow noreferrer">https://medium.com/read-write-code/these-are-the-autograders-you-ve-been-looking-for-bda0fa8fd8a</a></p>
  
  <p>You can also see the documentation for our autograder framework <a href="https://static.codehs.com/gulp/543c38430f2c5b3629343f8fc4d25596c12303c6/javadoc/autograders/index.html" rel="nofollow noreferrer">here</a></p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-06-13T21:15:11.967Z
Answers.LastActivityDate: 2017-06-29T19:28:38.867Z
Answers.LastEditDate: 2017-06-29T19:28:38.867Z
Answers.UserDisplayName: Zach Galant
Answers.Body: <p>For simple stuff I've used <a href="http://codingbat.com" rel="noreferrer">CodingBat</a>. They've got an <a href="http://codingbat.com/author" rel="noreferrer">authoring system</a> buried on the site. It's not terribly difficult to create your own questions, although the <a href="http://codingbat.com/authoring.html" rel="noreferrer">instructions are pretty wordy</a>.</p>
  
  <p>When you create a problem there are a set of fields to fill in. I'd attach a screenshot, but the network I'm on blocks imgur. I'll try to come back and add an image later.</p>
  
  <p><strong>doc</strong> </p>
  
  <p>This is the instructions that your students will read. </p>
  
  <p><strong>hint</strong> </p>
  
  <p>If you want your students to have the hint button, add some text</p>
  
  <p><strong>cases</strong> </p>
  
  <p>List as many test cases as you want. Separate each parameter that you want to send into the method you're testing with a comma. The expected answer comes at the end. </p>
  
  <p>It looks like this for the method below. <code>123</code> goes into <code>sumEvenDigits</code> as n and then <code>2</code> is the expected answer. </p>
  
  <pre><code>123,2
  2006,8
  </code></pre>
  
  <p><strong>code</strong>
  Starter code for your students. </p>
  
  <pre><code>public int sumEvenDigits(int n) {
  
  }
  </code></pre>
  
  <p><strong>tags</strong>
  Let's you put questions into categories, specify language, and set a difficulty. </p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-05-23T18:24:04.903Z
Answers.Text: I've looked for this before, but I never found it.  In your experience, practically speaking, how complex can an assignment be to be realistically tenable here?
Answers.UserDisplayName: Ben I.
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-05-23T18:30:52.530Z
Answers.Text: Works pretty well for warm up type stuff in an intro class. Not all that useful for labs once you get past single methods. I actually wound up writing my own because I couldn't do as much as I needed to with CodingBat.
Answers.UserDisplayName: Ryan Nutt
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-05-23T18:35:40.780Z
Answers.Text: Is your tool published?  *folds hands behind back, bats eyelashes innocently, whistles and looks down at feet*
Answers.UserDisplayName: Ben I.
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-05-23T18:38:48.437Z
Answers.Text: There's a link on my profile. Don't want to post a link here. Seems a little spammy :)
Answers.UserDisplayName: Ryan Nutt
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-05-23T18:09:42.113Z
Answers.LastActivityDate: 2017-05-23T18:09:42.113Z
Answers.UserDisplayName: Ryan Nutt
Answers.Body: <p>I would recommend <a href="https://repl.it/site/classrooms" rel="nofollow noreferrer">repl.it classroom</a> It allows creation of a course and assignments within that course. Assignments can be auto-graded using unit testing (JUnit) or by providing inputs and comparing expected outputs against actual outputs. Students get feedback on what test cases passed and which did not. These can then be converted directly into student grades.</p>
  
  <p>Here's a screenshot of one CodingBat problem:</p>
  
  <p><a href="https://i.sstatic.net/K9taG.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/K9taG.png" alt="repl.it classroom building an assignment"></a></p>
  
  <p>Repl.it also has the advantage of being completely cloud based -- no IDEs to install locally. Teachers and students both get free accounts. Java is just one of a dozen or more supported languages (Python, JavaScript, PHP, etc).</p>
  
  <p>The developers are really responsive -- I've had several conversations with Amjad Masad on Twitter -- and they take feedback seriously.</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-06-29T22:54:16.987Z
Answers.LastActivityDate: 2018-02-03T12:44:06.340Z
Answers.LastEditDate: 2018-02-03T12:44:06.340Z
Answers.UserDisplayName: Todd Whittaker
Answers.Body: <p>I use <a href="http://vpl.dis.ulpgc.es/" rel="nofollow noreferrer">VPL</a> (Virtual Programming Lab) for Moodle. In my case, it is just used for the introductory PHP classes (since HTML and DB interactions aren't possible), but it supports a wide range of languages, including Java. If you have Moodle available in your school, it is just a matter of installing the plugin. Then you're able to create your own tasks and define test cases in plain-text files. Everything is available from the activity interface itself. If you know a bit of shell script, it is even possible to perform more checks on the code, including style checks (you can even deny a submission if it doesn't match your criteria).</p>
  
  <p>Here is an example of a test cases file for a program that sums 2 values:</p>
  
  <pre><code>case = 1
  input =1
  2
  output =3
  case = 2
  input =3
  4
  output =7
  case = 3
  input =5
  6
  output =11
  case = 4
  input =-1
  -2
  output =-3
  case = 5
  input =-2
  1
  output =-1
  case = 6
  input =0
  6
  output =6
  case = 7
  input =7
  0
  output =7
  case = 8
  input =-10
  6
  output =-4
  case = 9
  input =10
  -9
  output =1
  case = 10
  input =100
  -200
  output =-100
  </code></pre>
  
  <p>VPL also supports automatic file comparison for plagiarism detection.</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-06-06T03:39:52.590Z
Answers.LastActivityDate: 2017-06-06T12:38:24.567Z
Answers.LastEditDate: 2017-06-06T12:38:24.567Z
Answers.UserDisplayName: rcpinto
Answers.Body: <p>The textbook I use (Horstmann's Java for Everyone) has an autograder that the author has written. He has posted many of the exercises from his textbooks. You can find it at:
  <a href="http://horstmann.com/codecheck" rel="noreferrer">Cay Horstmann's CodeCheck autograder</a></p>
  
  <p>There is an <a href="http://horstmann.com/codecheck/authoring.html" rel="noreferrer">authoring page</a> that shows how to write your own problems. I've found this to be the easiest of all the tools I've tried when it comes to creating my own assignments. </p>
  
  <p>The report page that is displayed when the students solve a problem includes the source code. Students may download a jar file with the results and submit that, but I just find it easier to have them print to pdf using their browser, and then upload that to our learning management system (Canvas). I can then browse through their code relatively quickly (although I still have to manually type in the grades while I'm doing that.)</p>
  
  <p>Someone has written an <a href="https://bitbucket.org/danielmai/code-check-homework-grading" rel="noreferrer">Grading tool for CodeCheck and Canvas</a> that downloads the uploaded jar files, allows you to comment on them locally and then uploads the grades back to Canvas. I found, however, that simply browsing the PDF printouts was more effective for me. </p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-03-15T17:42:01.167Z
Answers.Text: I've used CodeCheck myself for some years now, and it was particularly useful during the covid-19 pandemic.  A particular feature is worth mentioning: it does not need users (students or instructors) to register!  Everything works with randomly generated URLs.  Nevertheless, you can also integrate it with an LMS through LTI.  I've used it in Moodle and OpenEdX.
Answers.UserDisplayName: João Manuel Rodrigues
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-06-29T22:05:10.253Z
Answers.LastActivityDate: 2017-06-29T22:05:10.253Z
Answers.UserDisplayName: Stephen Gilbert
Answers.Body: <blockquote>
    <p>Disclaimer: I am one of the authors of this <strong>free</strong> service.</p>
  </blockquote>
  
  <p><strong>TL;DR: Use <a href="http://www.autogradr.com" rel="nofollow noreferrer">AutoGradr.com</a> which has been battle tested in university classrooms</strong></p>
  
  <hr>
  
  <ul>
  <li>AutoGradr lets you create your own labs and projects</li>
  <li>Labs: small, bite-sized questions to help students get familiar to a new concept. Students work in AutoGradr's web IDE and get instant feedback.</li>
  <li>Projects: suitable for more complex questions where students can work in their own IDEs (Visual Studio, Eclipse etc.). They upload their solutions to AutoGradr and get instant feedback.</li>
  <li>AutoGradr provides instant feedback to students allowing them to know their mistakes and make a better submission before it ever gets to you.</li>
  <li>AutoGradr provides report on code quality to both students and instructors. Students can make newer attempts to improve their submission.</li>
  <li>Instructors never have to write code to automate the grading.</li>
  </ul>
  
  <hr>
  
  <p>AutoGradr projects have test cases that define what the program output must be. Test cases can either be visible or hidden. </p>
  
  <p>Test cases can contain user input via stdin or files. Students can write to files and have them automatically tested.</p>
  
  <p><a href="https://i.sstatic.net/4QVZ3.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/4QVZ3.png" alt="Project on AutoGradr"></a></p>
  
  <hr>
  
  <p>AutoGradr's instant feedback tells student how their output differed from the expected output. They can make multiple attempts to fix any issues.</p>
  
  <p><a href="https://i.sstatic.net/tFWWR.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/tFWWR.png" alt="AutoGradr Feedback"></a></p>
  
  <hr>
  
  <p>Students and instructors can see report on code quality. Students can fix their code quality issues and make newer submissions.</p>
  
  <p><a href="https://i.sstatic.net/jRbe2.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/jRbe2.png" alt="AutoGradr Lint Report"></a></p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-04-06T13:56:53.397Z
Answers.Text: The link <http://www.autogradr.com/> seems to be dead.  Has this project been discontinued?
Answers.UserDisplayName: João Manuel Rodrigues
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-06-29T00:01:51.087Z
Answers.LastActivityDate: 2017-06-29T00:06:00.907Z
Answers.LastEditDate: 2017-06-29T00:06:00.907Z
Answers.UserDisplayName: tusharsoni
Answers.Body: <p>That seem an  important question.   Here is my input.
  'm using repl.it for a month, created ~60 assignments in 10 classrooms (java,c#).
  (Both basics, and also  recursion, lined-lists and Queues. ) </p>
  
  <p>The idea of web-based-ide &amp; auto-checking &amp; status-report-to-teacher is <strong>fantastic</strong> ! 
  It gets students very fast to code, and see results, also teacher knows immediately the situation of the class.
  Also some students find it hard to concentrate and listen to me, so they can go do tasks. 
  It is very easy to learn to use repl both for teacher and student.</p>
  
  <p>Teacher can create testbench and that saves a lot of time to students, so they can focus on their tasks, and get better testing to their code. Such testbench can include creating of various  trees/arrays/linked lists , so students can focus on the method they need to write, and get a lot done automatically for them.
  It is easy to student to figure out what teacher wanted in a task, since she can see the testcases ! ( so it saves teacher communication time!).</p>
  
  <p>We randomly divided a 30 HS-students-class in half, 15 with me and 15 with another teacher, my class using  repl.it, had progressed <em>much</em> faster, compared to the others that used traditional ways.</p>
  
  <p>**** repl.it  is both great and at the same time, far from great ****</p>
  
  <p>repl.it opened my eyes, now i want more from it. 
  There are details that make the experience  sub-optimal :
  -)  test-cases need to be entered using many gui clicks,  why can't there be <em>one</em> file,with many lines  input/output (see VPL-Moodle) ?<br>
  -)  Small tasks like renaming an assignment, accessing my solution, publishing, take way too many clicks, and between clicks we need to wait for web. 
  -) no source control,  if you by mistake clicked 'delete' for a test case it is gone  forever.Same for changes to source code.
  -)  hard to maintain assignments/classrooms over time, in collaboration with other teachers. 
  Guess there should be text files (under wiki/github/jenkins etc.), that contains <em>all</em> assignment info (question text, start/test-bench code visible to students, testcases inputs and outputs , solution ) ,  that can be put to other teachers review and improve, like an open source or a wiki  project.
  - hard  to maintain common files (teacher utils) common to all assignments  </p>
  
  <p>I think that there is going to be some serious money in "web-based-ide &amp; auto-checking &amp; status-report-to-teacher"   and would like to see some aggressive competition.
  Currently the only tool i know is repl.it.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-09-14T19:47:59.430Z
Answers.LastActivityDate: 2019-09-14T19:47:59.430Z
Answers.UserDisplayName: jsonphy
Body: <p>Grading currently either takes me a huge amount of time, or gets done in an extremely cursory way.  It occurs to me that, if my students were able to submit code into some sort of autotester, they would find many of their errors before I review their code.  This would mean that I would only have to look for clarity issues, not logic bugs.</p>
  
  <p>I am interested in software that (a) isn't too hard to set up, (b) will allow my students to test whether their code is (at least) to spec, and (c) allows me to quickly cycle through the code from all of the students in a section to look for style issues.</p>
  
  <p>I have looked far and wide, and have not yet found a reasonable system that accomplishes these three goals.  Many universities seem to have robust systems, but they all appear to be home-grown, and unavailable to outsiders.  Is anyone aware of such a system?  I honestly don't even care if it costs money - I just need a solution that makes this task more reasonable.</p>
  
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-06-30T13:50:27.547Z
Comments.Text: What type of programming are you testing? Small methods with well-defined inputs and outputs? Client-server programs such as websites? Graphical user interfaces in Swing? Each of these has to be tested differently...
Comments.UserDisplayName: tucuxi
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-05-23T14:22:59.063Z
Comments.Text: Software recommendations are hard on SE, but I could see this being useful to this community.
Comments.UserDisplayName: called2voyage
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-05-23T14:34:40.610Z
Comments.Text: very useful. This is a huge concern.
Comments.UserDisplayName: ncmathsadist
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-05-23T14:36:16.600Z
Comments.Text: To note it, there's an [SE for Software Recommendations](https://softwarerecs.stackexchange.com).  But, I'm not suggesting that this is off-topic since it seems like a helpful type of question.
Comments.UserDisplayName: Nat
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-05-23T14:36:25.217Z
Comments.Text: I agree that this question can be problematic for SE, but it is also extremely important.  How do other SEs deal with such list-format questions?  Surely there must be some mechanism on [matheducators](https://matheducators.stackexchange.com/).
Comments.UserDisplayName: Ben I.
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-05-23T14:50:08.280Z
Comments.Text: @Choirbean This is how most sites handle it: [Meta Math Educators](https://matheducators.meta.stackexchange.com/questions/101/books-and-materials-recommendations). They typically say "focused" but have their own cultural way of determining what focused means.
Comments.UserDisplayName: called2voyage
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-05-23T15:11:24.863Z
Comments.Text: I see this as a GOOD question about tool and workflow with a splash of pedagogy. For that reason I'd find it more valuable to keep this question here on the CS Ed site rather than on SE for Software Recs.
Comments.UserDisplayName: andytilia
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-05-25T17:51:37.593Z
Comments.Text: I have been considering Unit test tools (this is a class of tool, not a specific tool), so could be applied to other languages). I have just started teaching, so not tried yet. For Java I think JUnit is a popular one.
Comments.UserDisplayName: ctrl-alt-delor
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-05-30T04:47:56.947Z
Comments.Text: I'm curious to hear more on this topic. Almost tempted to start a bounty. I came across this thread regarding unit testing student work: https://stackoverflow.com/questions/28862949/how-to-junit-test-multiple-java-files-in-eclipse I'd like to hear what else people implement to automate the process. I do love CodingBat, but it would not work offline nor would it work if I don't first set everything up on their system.
Comments.UserDisplayName: Peter
Tags: <java><grading><auto-graders>
Title: What autograding software do folks use for Java code?
ThreadText: # What autograding software do folks use for Java code?
  
  (Asked by: username_0 on 2017-05-23)
  
  Grading currently either takes me a huge amount of time, or gets done in an extremely cursory way. It occurs to me that, if my students were able to submit code into some sort of autotester, they would find many of their errors before I review their code. This would mean that I would only have to look for clarity issues, not logic bugs.
  
  I am interested in software that (a) isn't too hard to set up, (b) will allow my students to test whether their code is (at least) to spec, and (c) allows me to quickly cycle through the code from all of the students in a section to look for style issues.
  
  I have looked far and wide, and have not yet found a reasonable system that accomplishes these three goals. Many universities seem to have robust systems, but they all appear to be home-grown, and unavailable to outsiders. Is anyone aware of such a system? I honestly don't even care if it costs money - I just need a solution that makes this task more reasonable.
  
  **Tags:** java, grading, auto-graders
  
  > **Comments:**
  > **username_2**: Software recommendations are hard on SE, but I could see this being useful to this community.  
  > **username_3**: very useful. This is a huge concern.  
  > **username_4**: To note it, there's an [SE for Software Recommendations](https://softwarerecs.stackexchange.com).  But, I'm not suggesting that this is off-topic since it seems like a helpful type of question.  
  > **username_0**: I agree that this question can be problematic for SE, but it is also extremely important.  How do other SEs deal with such list-format questions?  Surely there must be some mechanism on [matheducators](https://matheducators.stackexchange.com/).  
  > **username_2**: @Choirbean This is how most sites handle it: [Meta Math Educators](https://matheducators.meta.stackexchange.com/questions/101/books-and-materials-recommendations). They typically say "focused" but have their own cultural way of determining what focused means.  
  > **username_5**: I see this as a GOOD question about tool and workflow with a splash of pedagogy. For that reason I'd find it more valuable to keep this question here on the CS Ed site rather than on SE for Software Recs.  
  > **username_6**: I have been considering Unit test tools (this is a class of tool, not a specific tool), so could be applied to other languages). I have just started teaching, so not tried yet. For Java I think JUnit is a popular one.  
  > **username_7**: I'm curious to hear more on this topic. Almost tempted to start a bounty. I came across this thread regarding unit testing student work: https://stackoverflow.com/questions/28862949/how-to-junit-test-multiple-java-files-in-eclipse I'd like to hear what else people implement to automate the process. I do love CodingBat, but it would not work offline nor would it work if I don't first set everything up on their system.  
  > **username_1**: What type of programming are you testing? Small methods with well-defined inputs and outputs? Client-server programs such as websites? Graphical user interfaces in Swing? Each of these has to be tested differently...  
  
  ---
  
  ## 8 answers
  
  ### Answer by username_10 on 2017-05-23. Score: 9
  
  For simple stuff I've used [CodingBat](http://codingbat.com). They've got an [authoring system](http://codingbat.com/author) buried on the site. It's not terribly difficult to create your own questions, although the [instructions are pretty wordy](http://codingbat.com/authoring.html).
  
  When you create a problem there are a set of fields to fill in. I'd attach a screenshot, but the network I'm on blocks imgur. I'll try to come back and add an image later.
  
  **doc**
  
  This is the instructions that your students will read.
  
  **hint**
  
  If you want your students to have the hint button, add some text
  
  **cases**
  
  List as many test cases as you want. Separate each parameter that you want to send into the method you're testing with a comma. The expected answer comes at the end.
  
  It looks like this for the method below. `123` goes into `sumEvenDigits` as n and then `2` is the expected answer.
  
  ```
  123,2
  2006,8
  
  ```
  
  **code**
  Starter code for your students.
  
  ```
  public int sumEvenDigits(int n) {
  
  }
  
  ```
  
  **tags**
  Let's you put questions into categories, specify language, and set a difficulty.
  
  > **Comments:**
  > **username_0**: I've looked for this before, but I never found it.  In your experience, practically speaking, how complex can an assignment be to be realistically tenable here?  
  > **username_10**: Works pretty well for warm up type stuff in an intro class. Not all that useful for labs once you get past single methods. I actually wound up writing my own because I couldn't do as much as I needed to with CodingBat.  
  > **username_0**: Is your tool published?  *folds hands behind back, bats eyelashes innocently, whistles and looks down at feet*  
  > **username_10**: There's a link on my profile. Don't want to post a link here. Seems a little spammy :)  
  
  ---
  
  ### Answer by username_11 on 2017-06-29. Score: 6
  
  I would recommend [repl.it classroom](https://repl.it/site/classrooms) It allows creation of a course and assignments within that course. Assignments can be auto-graded using unit testing (JUnit) or by providing inputs and comparing expected outputs against actual outputs. Students get feedback on what test cases passed and which did not. These can then be converted directly into student grades.
  
  Here's a screenshot of one CodingBat problem:
  
  [![repl.it classroom building an assignment](https://i.sstatic.net/K9taG.png)](https://i.sstatic.net/K9taG.png)
  
  Repl.it also has the advantage of being completely cloud based -- no IDEs to install locally. Teachers and students both get free accounts. Java is just one of a dozen or more supported languages (Python, JavaScript, PHP, etc).
  
  The developers are really responsive -- I've had several conversations with Amjad Masad on Twitter -- and they take feedback seriously.
  
  
  ---
  
  ### Answer by username_12 on 2017-06-06. Score: 5
  
  I use [VPL](http://vpl.dis.ulpgc.es/) (Virtual Programming Lab) for Moodle. In my case, it is just used for the introductory PHP classes (since HTML and DB interactions aren't possible), but it supports a wide range of languages, including Java. If you have Moodle available in your school, it is just a matter of installing the plugin. Then you're able to create your own tasks and define test cases in plain-text files. Everything is available from the activity interface itself. If you know a bit of shell script, it is even possible to perform more checks on the code, including style checks (you can even deny a submission if it doesn't match your criteria).
  
  Here is an example of a test cases file for a program that sums 2 values:
  
  ```
  case = 1
  input =1
  2
  output =3
  case = 2
  input =3
  4
  output =7
  case = 3
  input =5
  6
  output =11
  case = 4
  input =-1
  -2
  output =-3
  case = 5
  input =-2
  1
  output =-1
  case = 6
  input =0
  6
  output =6
  case = 7
  input =7
  0
  output =7
  case = 8
  input =-10
  6
  output =-4
  case = 9
  input =10
  -9
  output =1
  case = 10
  input =100
  -200
  output =-100
  
  ```
  
  VPL also supports automatic file comparison for plagiarism detection.
  
  
  ---
  
  ### Answer by username_13 on 2017-06-29. Score: 5
  
  The textbook I use (Horstmann's Java for Everyone) has an autograder that the author has written. He has posted many of the exercises from his textbooks. You can find it at:
  [Cay Horstmann's CodeCheck autograder](http://horstmann.com/codecheck)
  
  There is an [authoring page](http://horstmann.com/codecheck/authoring.html) that shows how to write your own problems. I've found this to be the easiest of all the tools I've tried when it comes to creating my own assignments.
  
  The report page that is displayed when the students solve a problem includes the source code. Students may download a jar file with the results and submit that, but I just find it easier to have them print to pdf using their browser, and then upload that to our learning management system (Canvas). I can then browse through their code relatively quickly (although I still have to manually type in the grades while I'm doing that.)
  
  Someone has written an [Grading tool for CodeCheck and Canvas](https://bitbucket.org/danielmai/code-check-homework-grading) that downloads the uploaded jar files, allows you to comment on them locally and then uploads the grades back to Canvas. I found, however, that simply browsing the PDF printouts was more effective for me.
  
  > **Comments:**
  > **username_14**: I've used CodeCheck myself for some years now, and it was particularly useful during the covid-19 pandemic.  A particular feature is worth mentioning: it does not need users (students or instructors) to register!  Everything works with randomly generated URLs.  Nevertheless, you can also integrate it with an LMS through LTI.  I've used it in Moodle and OpenEdX.  
  
  ---
  
  ### Answer by username_15 on 2017-06-29. Score: 3
  
  > Disclaimer: I am one of the authors of this **free** service.
  
  **TL;DR: Use [AutoGradr.com](http://www.autogradr.com) which has been battle tested in university classrooms**
  
  ---
  
  * AutoGradr lets you create your own labs and projects
  * Labs: small, bite-sized questions to help students get familiar to a new concept. Students work in AutoGradr's web IDE and get instant feedback.
  * Projects: suitable for more complex questions where students can work in their own IDEs (Visual Studio, Eclipse etc.). They upload their solutions to AutoGradr and get instant feedback.
  * AutoGradr provides instant feedback to students allowing them to know their mistakes and make a better submission before it ever gets to you.
  * AutoGradr provides report on code quality to both students and instructors. Students can make newer attempts to improve their submission.
  * Instructors never have to write code to automate the grading.
  
  ---
  
  AutoGradr projects have test cases that define what the program output must be. Test cases can either be visible or hidden.
  
  Test cases can contain user input via stdin or files. Students can write to files and have them automatically tested.
  
  [![Project on AutoGradr](https://i.sstatic.net/4QVZ3.png)](https://i.sstatic.net/4QVZ3.png)
  
  ---
  
  AutoGradr's instant feedback tells student how their output differed from the expected output. They can make multiple attempts to fix any issues.
  
  [![AutoGradr Feedback](https://i.sstatic.net/tFWWR.png)](https://i.sstatic.net/tFWWR.png)
  
  ---
  
  Students and instructors can see report on code quality. Students can fix their code quality issues and make newer submissions.
  
  [![AutoGradr Lint Report](https://i.sstatic.net/jRbe2.png)](https://i.sstatic.net/jRbe2.png)
  
  > **Comments:**
  > **username_14**: The link <http://www.autogradr.com/> seems to be dead.  Has this project been discontinued?  
  
  ---
  
  ### Answer by username_16 on 2019-09-14. Score: 2
  
  That seem an important question. Here is my input.
  'm using repl.it for a month, created ~60 assignments in 10 classrooms (java,c#).
  (Both basics, and also recursion, lined-lists and Queues. )
  
  The idea of web-based-ide & auto-checking & status-report-to-teacher is **fantastic** !
  It gets students very fast to code, and see results, also teacher knows immediately the situation of the class.
  Also some students find it hard to concentrate and listen to me, so they can go do tasks.
  It is very easy to learn to use repl both for teacher and student.
  
  Teacher can create testbench and that saves a lot of time to students, so they can focus on their tasks, and get better testing to their code. Such testbench can include creating of various trees/arrays/linked lists , so students can focus on the method they need to write, and get a lot done automatically for them.
  It is easy to student to figure out what teacher wanted in a task, since she can see the testcases ! ( so it saves teacher communication time!).
  
  We randomly divided a 30 HS-students-class in half, 15 with me and 15 with another teacher, my class using repl.it, had progressed *much* faster, compared to the others that used traditional ways.
  
  \*\*\*\* repl.it is both great and at the same time, far from great \*\*\*\*
  
  repl.it opened my eyes, now i want more from it.
  There are details that make the experience sub-optimal :
  -) test-cases need to be entered using many gui clicks, why can't there be *one* file,with many lines input/output (see VPL-Moodle) ?  
  -) Small tasks like renaming an assignment, accessing my solution, publishing, take way too many clicks, and between clicks we need to wait for web.
  -) no source control, if you by mistake clicked 'delete' for a test case it is gone forever.Same for changes to source code.
  -) hard to maintain assignments/classrooms over time, in collaboration with other teachers.
  Guess there should be text files (under wiki/github/jenkins etc.), that contains *all* assignment info (question text, start/test-bench code visible to students, testcases inputs and outputs , solution ) , that can be put to other teachers review and improve, like an open source or a wiki project.
  - hard to maintain common files (teacher utils) common to all assignments
  
  I think that there is going to be some serious money in "web-based-ide & auto-checking & status-report-to-teacher" and would like to see some aggressive competition.
  Currently the only tool i know is repl.it.
  
  
  ---
  
  ### Answer by username_8 on 2017-06-29. Score: 1
  
  We've been using [Web-CAT](http://web-cat.org). It's ok. It certainly motivates students to read error messages more closely.
  
  EDIT: Here are three things that have bugged me about Web-CAT:
  
  * I remember having difficulty determining exactly where my unit tests were supposed to be uploaded; there's a user folder and an assignment folder, and one something's in the wrong place it's hard to just move it.
  * In three years, I still haven't found how to get it to accept "Type arrayname[]" as opposed to "Type[] arrayname"; students hate losing those two points.
  * Students sometimes have to click around a little to find report details, and the UI doesn't always make clear what's clickable.
  
  > **Comments:**
  > **username_0**: Hi username_8!  This is something of a stub answer.  Would you mind editing it?  Don't worry, the norms of answering on SE sites are not always obvious, and you're definitely not the first one to have a problem like this :) What can you tell us about Web-CAT?  What are its strong points and its weaker points?  
  > **username_8**: Thanks for the note.  I'll get the hang of this.  :)  1) I remember having difficulty determining exactly where my unit tests were supposed to be uploaded; there's a user folder and an assignment folder, and one something's in the wrong place it's hard to just move it.  2) In three years, I still haven't found how to get it to accept "Type arrayname[]" as opposed to "Type[] arrayname"; students hate losing those two points.  3) Students sometimes have to click around a little to find report details, and the UI doesn't always make clear what's clickable.  
  > **username_0**: Oh, I meant in the answer itself :) Comments are considered temporary content, even if in practice they rarely get deleted.  
  > **username_8**: I only just now realized I can edit my answer!  Thanks!  
  
  ---
  
  ### Answer by username_9 on 2017-06-13. Score: 0
  
  > Note: I'm one of the founders of CodeHS.
  
  If you check out CodeHS, we have a system for writing autograders using Java. You can write unit test specific problems that test the output of a specific method or write generic autograders for any Java program if you want to test more complex things.
  
  You can read more about them here: <https://medium.com/read-write-code/these-are-the-autograders-you-ve-been-looking-for-bda0fa8fd8a>
  
  You can also see the documentation for our autograder framework [here](https://static.codehs.com/gulp/543c38430f2c5b3629343f8fc4d25596c12303c6/javadoc/autograders/index.html)
  
  
  ---
  
  