Ink Over Video

I am trying to allow the user to draw on top of a mediaelement(video) when the clip is paused. I have tried overlaying an inkcanvas over the mediaelement and also placing the mediaelement as a child of an inkcanvas. Using both methods, when the user draws on the video it tears (the underlying video isn't refreshing the current frame). The best solution I have come up with is grabbing the current frame as a bitmap when the user pauses and overlaying that to collect the ink. I would much prefer to not due this for timing issues and it causes some jittering on resuming playback. I have included my frame overlay code below, but I would greatly appreciate any suggestions for a better approach. Thanks.

--Brian

RenderTargetBitmap tg = new RenderTargetBitmap((int)MainVideo.ActualWidth, (int)MainVideo.ActualHeight, 96, 96, PixelFormats.Default);
tg.Render(MainVideo);
OverlayImage.Source = tg;
OverlayImage.Width = MainVideo.ActualWidth;
OverlayImage.Height = MainVideo.ActualHeight;
OverlayImage.Visibility =
Visibility.Visible;



Answer this question

Ink Over Video

  • Mr-T

    I haven't and that's a good suggestions. I will try it and report back, thanks!
  • Marko85

    That is a really COOL application. But How do we save ink drawn on Video/Media Element


  • brendon.smith

    Just as an experiment have you tried using a VisualBrush to display the MediaElement under / inside the InkCanvas It may not work any better than using the media element itself, but on the other hand it might.

  • Pavan KumarKavuta

    First you must have an InkCanvas as the parent of the MediaElement or on top of the MediaElement.

    After that saving the ink is pretty straightforward, call inkCanvas.Strokes.Save(strm) where inkCanvas is the InkCanvas element and strm is a Stream object. The stream can be any stream from a MemoryStream to a FileStream. This will serialize and save the strokes (ink) to the stream which can later be loaded via the StrokeCollection constructor overload which takes a Stream as a argument which can then be loaded into the InkCanvas.


  • Skippy1313

    Apparently there is a bug with some NVidia drivers... hopefully this will get better with Beta2.

    Do you have this problem with any other hardware (esp. non-NVidia systems)



  • RyanTF

    well I tried moving the mediaelement offscreen(and also just moving the mediaelement down in z-order) and using a visualbrush to paint the Background of the InkCanvas. It plays the video just fine, but still has the tearing issues. I tried a couple different ways, but couldn't get it to work. If you get a working code sample, please let me know, but I was not able to. It appears that when a mediaelement is pauses it just doesn't repaint that area as far as I can tell
  • TazInfo

    Thanks, let me know as soon as you know something because this is holding up some of our development. I think this also effects functionality such as frame-by-frame advance of video while paused. It seems due to lack of redraw when invalidated and paused.
  • redwyre

    I put together a small sample which illustrates the problem I'm having. In my system many of the objects are generated programatically and I use a MediaClock to control it, but the effect is the same. In order to repro: 1) replace the video location with the location of a valid video file 2) run the app and play the video 3) Pause the video anywhere during playback 4) Attempt to draw ink on the video and notice the tearing. I have also captured a screen shot of the tearing if that would help at all. I am running this on XP sp2 on an Acer C200 tablet with a GeForce Go 6200 with 64meg ram. It is running in hardware as verified with perforator. Thanks for your help and please let me know if you need anything else or more information.

    Code Sample:

    <StackPanel
    xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
    xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
    xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006"
    xmlns:d="http://schemas.microsoft.com/expression/interactivedesigner/2006"
    mc:Ignorable="d"
    Background="#FFFFFFFF"
    x:Name="DocumentRoot"
    Width="640" Height="480">
    <StackPanel.Resources>

    </StackPanel.Resources>

    <StackPanel.Triggers>
    <EventTrigger RoutedEvent="Button.Click" SourceName="PlayButton">
    <EventTrigger.Actions>
    <BeginStoryboard x:Name="MainVideoTimeLine" >
    <Storyboard>
    <MediaTimeline Source="D:\fb_videos\test.avi" Storyboard.TargetName="MainVideo"
    BeginTime="0:0:0" />
    </Storyboard>
    </BeginStoryboard>
    </EventTrigger.Actions>
    </EventTrigger>
    <EventTrigger RoutedEvent="Button.Click" SourceName="PauseButton">
    <EventTrigger.Actions>
    <PauseStoryboard BeginStoryboardName="MainVideoTimeLine" />
    </EventTrigger.Actions>
    </EventTrigger>
    </StackPanel.Triggers>


    <InkCanvas Height="400" >
    <MediaElement Canvas.Left="0" Canvas.Right="0" x:Name="MainVideo" RenderTransformOrigin="0.5,0.5" Height="400" Width="600"/>
    </InkCanvas>

    <Button HorizontalAlignment="Center" x:Name="PlayButton">Play</Button>
    <Button HorizontalAlignment="Center" x:Name="PauseButton">Pause</Button>


    </StackPanel>


  • Jeff Pigott

    I hate to bring back this old of a post, but this problem still exists in Beta 2. I have tried this on several different computers. I have also tried running this on both Vista Beta 2 and Win XP sp2. Also using a Nvida based card or an ATI based card. Furthermore, the problem happens with wmv or mpeg4 files. I noticed that with different combinations of video cards and video formats the problem happens with different frequencies. For example, with Vista/ATI (x600)/WMV it only tears about 1/5 of the time. But with MPEG4 (xvid) it happens much more frequently. This might also be related to the seeking problems I am having. Whenever I seek in a video while it is paused, it doesn't update the screen until you play the graph again. This also effects frame-by-frame advance. All of these problems are while the video is paused -- not playing and not stopped. Thanks for your help.

    --Brian


  • fcampanini

    I just got it working in Vista and there are no tearing issues with the ATI x600 card. There are definitely still some performance and redraw issues with the video, but it looks like the tearing won't be and issues (at least with Vista). Thanks for your help on this and please let me know if there are any updates or if you know what is the problem with xp sp2.

    EDIT: In Vista it works about 5/8 times, but 3/8 times it still tears. I haven't been able to find a pattern yet. In addition, sometimes it won't start tearing until the middle of a stroke. I can email you screenshots if you want.

  • LisaF

    I just went out and got an ATI x600 256meg video card to test this out. Under XP sp2, I still get the same tearing issues with this video card as with the nvidia. I am trying to test it under Vista, but I just got the new 5365 build and I am having a couple issues getting winfx back up and running so I haven't been able to test it yet.

  • Makoto

    Hi Brian... do you have a small snippet of XAML you can share Ink over MediaElement should work fine, in Feb CTP.

    Is this on XP or Vista What video card does your system have -- or does the issue repro on systems with different hardware (MediaElement should fall-back to software rendering, on some systems... but it should still work.)

    -S



  • VS2005Simen

    If you want the MediaElement to update its frame when seeking in paused state, you need to set the ScrubbingEnabled flag on your MediaElement.

    Thanks a lot for your feedback,
    Ed Maia
    WPF PM


  • BethM

    Ah, inking while the video is paused Yes, I seem to recall there may be a bug there.

    Thanks for the detailed repro -- I'll make sure the right people take a look at it.

    -S



  • Ink Over Video