"In the visual effects industry, the term rotoscoping refers to the technique of manually creating a matte for an element on a live-action plate so it may be composited over another background." Wikipedia
This effect is based on the MLT FilterRotoscoping
Rotoscoping options greyed out
When you first add the rotoscope effect the keyframe options in it are greyed out. You can not even add a new keyframe.
Solution. Click in the project monitor and start drawing your rotoscope matte. Then the rotoscope keyframe options become enabled.
Examples of what you can do with the Rotoscoping effect.
see also Rotoscoping on ttill's blog
Tutorial with Rotoscoping
This video is about green screen on Kdenlive but he does use rotoscoping in it too.