Q:

Can data be decrypted with the public key if encrypted with the private key?

I am trying to understand the principles of asymmetric encryption, and have read the following in a book:

One key is completely public and can be read and used by everyone. The
  other part is private and should never be shared with someone else.
  When you encrypt something with the public key, it can be decrypted by
  using the private key, and vice versa.

Looks like the public key is not secret. But if I encrypt with the private key, someone can decrypt with the public key which is easily accessible. Where's the logic in that?

A:

Looks like public key is not secret. But if I encrypt with private, someone can decrypt with public which is easy accessible. Where is logic?

So let's go the public encrypt -> private decrypt route.  You're seemingly already aware of the use case of this.  Users are able to encrypt things with your public key, that only your private key can decrypt.  This is useful for when say a user on the internet wants to send a message that only you, as the owner of the private key, can decrypt.
This is all well and good, but what purpose does the other direction serve (encrypt w/ private -> decrypt w/ public)?  The answer is, you can help assure that you are a owner of the private key to a corresponding public key when you encrypt a message with your private key, and the public key is able to successfully decrypt and verify the message.  This is the basic idea around Signature Generation and verification.

A:

The usage of asymmetric keys is as follows:

Encryption is done using the public key
Signing is done using the private key

Encryption:

The data is encrypted using the public key
The encrypted data is decrypted using the private key

Only the owner of the private key can read the encrypted message

Signing

The HASH of the data is encrypted using the private key
The receiver use the public key on the encrypted hash data, and compare it to the the HASH of the data he calculate himself

Only the owner of the private key can sign the message

