Q:

Should I ever use exceptions in java if I can solve problem without them?

Are exceptions like goto in c don't use it if you don't have to or is it OK to use them as much you want to, and why?
Are they hard on CPU, memory or something else?

A:

...don't use it if you don't have to or is it OK to use them as much you want to...

According to Joshua Bloch's Effective Java: 

Exceptions are, as their name implies, to be used only for exceptional
  conditions; they should never be used for ordinary control flow.

In order to fully understand this, let's take a look on Java exception. In Java we can find three types of exceptions: checked exceptions, runtime exceptions, and
errors.
Checked exceptions: as their name implies, they should be checked/treated by the developer. Moreover, the compiler forces you to handle this exceptions by try...catch or propagating them upwards by marking the method with throws keyword. These kind of exceptions usually extend the builtin `Exception' class.
Unchecked exceptions: in this group enter both runtime exceptions and errors. The difference between checked exception and unchecked exceptions is that the compiler does not forces you to handle unchecked exceptions. Since they should be used to indicate programming errors, usually it is a good idea to not catch them at all and focus on preventing them. Errors are reserved for use by the JVM to indicate critical failures which usually prevent your program to be recoverable. All of the unchecked
exceptions should subclass RuntimeException class.
So, to give you a response to your first question, it is OK to use exceptions as long as it makes your code readable, more understandable. Moreover, you should be able to deal with exceptions, since compiler will force you to do so. This does not mean that you should throw exceptions whenever you want.
Now let's talk about performance. To repeat myself, exceptions should be used in exceptional cases. This means that they are not as performant as a single if..else would be. Using them wisely, you wont really notice a performance hit, overusing them can cause problems. Exceptions should never be used for control flow (gotos can be used for). In case of memory, every exception creates a new object, which should contain some information about the exception itself. Creating many exceptions would result in occupying more memory, but I don't think that this can be a problem nowadays.

Should I ever use exceptions in java if I can solve problem without
  them?

It depends, and is up to you on deciding against or in favor of them. Usually, exceptions are thrown to notify the lack of something (example: no money to withdraw from a credit card, non-existent file to open, etc.) In order to avoid using exceptions in these cases, it is a good idea to return on Optional of something.  
This answer is based on Joshua Bloch's Effective Java book. For more information I suggest reading the Exceptions chapter.

