Universal Web Scrubbing Experience

I was surprised that the first touch device hard to deal with is Edge browser on Windows tablet. I though it was just like chrome. It had been years that the only touch events I had to handle is TouchEvent. Most web applications have simple interactions: just click. If you can click with mouse, you can navigate, trigger popup or submit forms.

Think about it, while you're watching a video on tablet, or with your cell phone, you want to jump further seconds but there is only clicks can be achieved, will you feel disappoint? To protect user experience such as scrubbing the timeline or playhead (indicator) for seeking the position, we need scrub.

Ahhhh...Cross Browser?

Click is across browser, browser doesn't care about your mouse, but screen does care about your finger. Usually in the devices supporting touch events, it has touchstart, touchmove, touchend and touchcancel. But after chrome introducing pointer events, dev noticed that there were still ways to handle similar stuff like mouse events.

just change all mouse events prefix to pointer, you'll get pointerup, pointermove, etc. Also, with the ability to detect is fired by finger, mouse or a pen.

Some of our requirements have different behavior of scrubs on player timeline:

• when you do touch scrub, the indicator on timeline move, but mouse doesn't
• when you do mouse hover and keep moving, the indicator move as well

Hover vs touch scrubs. Here we gotta detect the type of input. How?

First Failed Try

According to the spec of pointer events on w3c:

You can see a ordered sequence of events firing. For example, when you do click, then following events got fired.

mousemove
pointerover
pointerenter
mouseover
mouseenter
pointerdown
mousedown
Zero or more pointermove and mousemove events, depending on movement of the pointer
pointerup
mouseup
click
pointerout
pointerleave
mouseout
mouseleave

Observing that the corresponding pointer events are fired before mouse events, every event starts with a mouseover. I want to handle all events inside one function so we decided that maintain the continuous pointer type during an finger/mouse interaction. Since Safari doesn't have pointer events, I decided to use TouchEvent as replacement. The we could just listen to mouse events, every mouse event instance expect first one will be set with a attribute pointerType to tell you the input source.

Actually, the idea that setting event with pointerType is came from Netflix's website. I reversed engineering some compressed js code (not fully mangled) I found the solution.

But finally i realize that won't work, 'cause mouse events are not always fired on touch devices. Netflix use native controls in Safari if watching a content without login. (I don't know what it will change if you login, but I guess it won't change too much? the code is there). They use a product level solution to resolve this friction.

Smart approach I'd say so, but my product didn't tell me to enable native video controls while user watching on our site. So I tested on different devices/browsers and made a list of status of events support. See:

Browser & Action	PointerEvent	MouseEvent	TouchEvent
chrome + touch	✅	✅	✅
edge + touch	✅	✅	❌
safari + touch	❌	❌	✅
chrome + mouse	✅	✅	❌
edge + mouse	✅	✅	❌
safari + mouse	❌	✅	❌

Is there any way to handle all these stuffs universally?

Cross Browser Scrub Solution

Scrubs have there types:

• scrub with finger touch
• scrub with mouse drag
• mouse over and move on

One behavior may happen on these two composed from three scenarios above , or event only one. Our approaching is to separate these actions by configurations, make them independent.

What Developer Needs?

Usually we only care about the triggered callbacks, start / move / end are enough. They could be designed as events or handlers.

How to Separate These Actions?

Let's go through the movements:

• Hovering elements with mouse: enter, move, leave
• Scrubbing through: pointer down / move / pointer up

that's it. Due to different status of events support, we need to handle them in proper way. The handlers may receive event which is instance of PointerEvent, MouseEvent or TouchEvent.

Source of Code

If you want to see the details of implementation check huozhi/fpoint. The API is pretty simple and easy to get started. You can also check the demo on different browsers/devices.

Feel free to submit issues or questions on the repo. :)