I was surprised that the first touch device hard to deal with is Edge browser on Windows tablet. I though it was just like chrome. It had been years that the only touch events I had to handle is
TouchEvent. Most web applications have simple interactions: just click. If you can click with mouse, you can navigate, trigger popup or submit forms.
Think about it, while you're watching a video on tablet, or with your cell phone, you want to jump further seconds but there is only clicks can be achieved, will you feel disappoint? To protect user experience such as scrubbing the timeline or playhead (indicator) for seeking the position, we need scrub.
Click is across browser, browser doesn't care about your mouse, but screen does care about your finger. Usually in the devices supporting touch events, it has
touchcancel. But after chrome introducing pointer events, dev noticed that there were still ways to handle similar stuff like mouse events.
just change all mouse events prefix to
pointer, you'll get
pointermove, etc. Also, with the ability to detect is fired by finger, mouse or a pen.
Some of our requirements have different behavior of scrubs on player timeline:
- when you do touch scrub, the indicator on timeline move, but mouse doesn't
- when you do mouse hover and keep moving, the indicator move as well
Hover vs touch scrubs. Here we gotta detect the type of input. How?
First Failed Try
According to the spec of pointer events on w3c:
You can see a ordered sequence of events firing. For example, when you do click, then following events got fired.
mousemove pointerover pointerenter mouseover mouseenter pointerdown mousedown Zero or more pointermove and mousemove events, depending on movement of the pointer pointerup mouseup click pointerout pointerleave mouseout mouseleave
Observing that the corresponding pointer events are fired before mouse events, every event starts with a
mouseover. I want to handle all events inside one function so we decided that maintain the continuous pointer type during an finger/mouse interaction.
Since Safari doesn't have pointer events, I decided to use
TouchEvent as replacement. The we could just listen to mouse events, every mouse event instance expect first one will be set with a attribute
pointerType to tell you the input source.
Actually, the idea that setting event with
pointerType is came from Netflix's website. I reversed engineering some compressed js code (not fully mangled) I found the solution.
But finally i realize that won't work, 'cause mouse events are not always fired on touch devices. Netflix use native controls in Safari if watching a content without login. (I don't know what it will change if you login, but I guess it won't change too much? the code is there). They use a product level solution to resolve this friction.
Smart approach I'd say so, but my product didn't tell me to enable native video controls while user watching on our site. So I tested on different devices/browsers and made a list of status of events support. See:
|Browser & Action| PointerEvent | MouseEvent | TouchEvent | |:---:|:---:|:---:|:---:| | chrome + touch | ✅ | ✅ | ✅ | | edge + touch | ✅ | ✅ | ❌ | | safari + touch | ❌ | ❌ | ✅ | | chrome + mouse | ✅ | ✅ | ❌ | | edge + mouse | ✅ | ✅ | ❌ | | safari + mouse | ❌ | ✅ | ❌ |
Is there any way to handle all these stuffs universally?
Cross Browser Scrub Solution
Scrubs have there types:
- scrub with finger touch
- scrub with mouse drag
- mouse over and move on
One behavior may happen on these two composed from three scenarios above , or event only one. Our approaching is to separate these actions by configurations, make them independent.
What Developer Needs?
Usually we only care about the triggered callbacks, start / move / end are enough. They could be designed as events or handlers.
How to Separate These Actions?
Let's go through the movements:
- Hovering elements with mouse: enter, move, leave
- Scrubbing through: pointer down / move / pointer up
that's it. Due to different status of events support, we need to handle them in proper way. The handlers may receive event which is instance of
Feel free to submit issues or questions on the repo. :)