Q:

Strength of RSA encryption with or without a public key

An RSA key is actually a key pair: a public key and a private key. The public key is usually uploaded to a server. If I understand correctly, in this situation, attacking RSA amounts to solving a math factoring problem, for which there are (currently theoretical) efficient quantum algorithms.
My question is, what if the RSA public key is not actually made public at all? Is it as easy to attack such "purely private" RSA encryption than it is to attack "normal" RSA encryption, in which the public key is known? Is the public key somehow embedded in the RSA encrypted message? If not, do the quantum algorithms need the RSA public key to work?
If purely private RSA is harder to attack, how does purely private 2048/4096 RSA encryption compare to 128/256 AES?
Edit: I understand that:

RSA is usually only used to encrypt a random "session key", which is then used for symmetric (say, AES) cryptography. E.g., in OpenPGP: https://tools.ietf.org/html/rfc4880#section-2.1 .
The extra RSA layer introduces an extra vulnerability: one can attack it by breaking either RSA or the session AES.
If the extra RSA layer uses a known public key, it is "much easier" to break that, at least in theory, using future quantum algorithms, than it is to break AES.

What I'm trying to understand is, how does "purely private" RSA + session AES compare with AES alone (assume the same AES key size).
Edit 2: To avoid unnecessary tangents, I added an extensive clarification of what I'm trying to understand here. The question there is "Is there a hacky way to remove the unnecessary RSA?". The OP here asks "Just how bad is the unnecessary RSA?"

A:

No widely used system uses RSA to encrypt data. RSA is slow. Really slow. RSA is used together with an symmetric cypher like AES. A random key is created for AES, the data is encrypted with AES and that key. Then the Key is encrypted with RSA and thrown away.
If you can crack the symmetric cypher (usually AES nowadays) you can also break the hybrid encryption. In practice, there is no advantage for you. In one case, the attacker has to break AES. In the other case, he can either break AES or RSA to get your message!
I don't think there is a thorough analysis of the RSA strength without the publik key known. It depends on the exact software/format you use, if the public key is embedded or not. It is quite possible to not embed it. But in the end, RSA just has no use case when it is not used as asymmetric cryptography. 
If RSA is secure and no fault is found, then it still can be decrypted if you have the public key and can factorize it. For AES this isn't true. If no flaw is found in AES, it cannot be cracked. It is just possible to brute-force attack it. This means trying all keys and check if a valid message is output. Depending on the format of your message, even deciding if the message is valid is very hard.
If you don't need asymmetric cryptography, use symmetric cryptography!

