Question: I understand that sampling rate should be twice the signal bandwidth for successful reconstruction. But what I don't get is how this translates in the time domain.

For example, I have a sine wave pulse with frequency 100MHz and duration of say 50ns. This signal has a bandwidth of 20MHz (1/50ns = 20MHz). Theoretically, I can sample this signal at any frequency above 40Mhz. Let's say I sample at 50MHz, then I only get 2 samples per pulse!! Now what can you possibly do with those 2 samples is my question. In other words, although Nyquist criterion is satisfied in this example, there's not enough data points to proceed.

This kind of signals are common in radar and generally you want to run some sort of matched filtering/correlation once you have digitized the pulse. But you can't run any descent signal processing on just two samples? am I missing something here?

Answer: 
I solved my problem. The design methodology as described above is sound. However, there was an error in my implementation of the FIR filter. For symmetrical filters you can replace convolution by correlation, and that's what I did. However, since the polyphase filters are asymmetrical this led to incorrect results. I implemented the FIR filter by convolution and now my sample rate converter works like a charm.

Below you can see the correctly resampled image:

![The correctly resampled image](https://i.stack.imgur.com/l76ew.png)

And here's the original input image:

![original image](https://i.stack.imgur.com/Ghcvb.png)
Your approach seems conceptually sound. I suspect that the problem is one of two things: 1) you have implemented it incorrectly, 2) Your filter bandwidth is too wide.

You are implementing the "smart" approach in that you are doing it the computationally efficient way. I would try doing it the dumb way (insert I-1 zeroes between every sample, filtering with the entire filter, get rid of the extra samples) to verify your implementation and to see if the filter itself is the problem. You can verify your implementation by comparing your resulting samples against the samples produced by the dumb approach. After you have filtered the upsampled data one of the sample "phases" (there are "D" phases whose start samples are samples 0...D-1) should be the same as the smart approach's results. If not, something is wrong in your implementation of either the smart or dumb approach.

If the results do match, then the problem is your filter. You need to look closely at the bandwidth to make sure that it isn't too wide. Since D > I the ideal filter bandwidth, assuming a perfect brick wall filter, is $\frac{I}{2D}$, where the factor of two comes from the Nyquist rate.