Question: Problem specification: 
I have a rectangular and uniformly spaced image of pixels with vertex coordinates (i,j), (i+1,j), (i, j+1), (i+1, j+1) [i=0,...,m-1; j=0,...,n-1] and a polygon P with vertex coordinates (x\_1,y\_1), ..., (x\_n, y\_n). Now I want to efficiently compute the percentage of every pixel overlapping with P. P can be non-convex, or even self-intersection.

Essentially, this is a "soft" generalization of the scan-line rasterization algorithms which check efficiently if the pixel centers lie inside / outside the polygon.

I can think of the following approaches:

(1) Upsample the image (e.g. by a factor 10\*10), count how many subpixel centers lie inside the polygon, and divide by 100. Problems: time efficiency, memory efficiency, accuracy.

(2) Use the scan-line algorithm on a slightly bigger and by (0.5,0.5) translated grid to compute the pixels that lie fully inside / outside, create a list of "borderline" pixels, walk counter-clockwise along the edges and compute the intersection areas with all pixels along the way. Problems: requires subtle coding, easy to introduce bugs.

My question: Has anybody already encountered this problem, and do you know a third, superior approach? And if not, have you made better experiences with (1) or with (2)? I assume that this problem may arise in the context of antialiasing?

Answer: 
Doing the exact geometric analysis might not be too difficult.

Deal with those pixels that are partially covered by the polygon first: you can use a [technique from ray-tracing](http://www.cse.yorku.ca/~amana/research/grid.pdf) to quickly find all pixels that intersect with the polygon edges. You can then use the [Cohen-Sutherland](http://en.wikipedia.org/wiki/Cohen%E2%80%93Sutherland_algorithm) algorithm to efficiently find the points of intersection between the edge and the pixel, and hence you can compute the area of coverage for that pixel.

Note that you can avoid one of the two clipping operations involved in Cohen-Sutherland as adjacent pixels will share a segment intersection point. For instance - if you have two adjacent pixels, `A` and `B` that intersect with a segment `p->q` at points `a1`, `a2`, `b1` and `b2`, then `a2` and `b1` will be the same. Passing the segment `a2->q` into the routine when clipping against `B` should avoid repeating work.

You'll have to treat the pixels that contain the polygon vertices specially, but again it shouldn't be too tricky: Cohen-Sutherland will help here as well.

Self-intersecting polygons will also throw up some special cases to handle - pixels that intersect with two or more edges. I can easily imagine that handling these exactly in all cases might get tricky, so I'd be tempted to just do the upsampling approach here.

Once these edge pixels have been identified, you can do the standard scan-line thing to fill in the polygon's interior pixels.

edit: Actually, now that I think more about it, you can totally skip the Cohen-Sutherland step. The algorithm in the linked paper can be easily extended to return the intersection points between the segment and the pixel grid. The segment will leave a given pixel at `min( tMaxX, tMaxY )`. Keep track of the last exit point to re-use as the entry point for the next pixel.
I would do 

1a) Upsample when the pixel is partly overlapping:  

but not the whole image, only the current pixel to be checked, or all pixels in the current scan line if that helps.  

Than there is no memory argument.   

speed? up to 16x16 i dont think that speed is an issue.