I've already used my entire dataset in a regression, should I not use that as a prediction model? At the hospital I work at we were writing a paper on what variables about a patient predict whether they'll return for a follow-up visit.  We included variables such as age, gender, distance from their home to the hospital, mechanism of injury and other things like that. We had about 600 patients to examine and so we ran a multiple logistic regression with yes/no return as the outcome, and we did this with everyone in our dataset (everyone with that condition at our hospital).
Well we wrote the paper and then someone decided we should try to create an online prediction tool. You could put in variables about a patient, and it would return a guess about whether or not the patient would return, based on our previous regression model. To help me create an online prediction tool I've used this tutorial using R and Shiny and I noticed the author split his data into training and testing sets
Problem is: I never did that. Reading comments such as this I think I understand why someone would split their data, but my question now is:
What can/should I do about it?

*

*I've already used all my data.  Would it be best to delete everything I've done, go back, split the data and start over?  (We didn't publish the paper or anything)

*Should I just proceed? Can an argument be made for NOT splitting the dataset?


In my opinion the best course of action, if possible, is to collect more data and then use that data to check your current model as well as maybe top 5 of the previous models you tried.
Continuing with the child learning multiplication example from the comment you referenced - each of your models is a different child. You set up a procedure which ranks children according to how well they perform multiplication on the data they have already seen. This procedure is biased towards those who memorised the table. The child who best learned how to multiply might rank (e.g.) third from the top or even lower. So the only way to select models that will perform well outside of the data you have so far is to put them to the test using a new set of data (suitable named "testing" data).
If getting more data is impossible you can always do cross-validation. But here you will have to re-estimate your models. You can learn more about cross-validation by looking at the relevant answers on this site, but the idea is to simulate training/testing splits while still using all of the training data.
If you cannot even adjust the original analysis then the last best thing might be to select a well-enough performing model that uses the least number of variables. For example, if one model reaches 76% accuracy using 30 variables, and another one reaches 72% wile using only 10 - it is less likely for the lesser model to have "memorized" the data. So we would expect that model to perform better on new patients.
