Files
palemoon27/dom/media/DOMMediaStream.cpp
T
roytam1 21c98793d3 import changes from `dev' branch of rmottola/Arctic-Fox:
- Bug 1244883 - Add Nightly-/Aurora-only crash for AsyncTransactionWaiter timeouts - r=nical (132b2ceff9)
- Bug 1148978 - Trigger paints when moving plugin windows around on the browser main thread. r=mattwoodrow (c75ce5ec09)
- Dedent some functions. (bug 1254899 part 1, r=jrmuizel) (c84fb419c7)
- Refactor acceleration pref initialization. (bug 1254899 part 2, r=jrmuizel) (29a164c70e)
- Remove NS_NATIVE_GRAPHIC on Windows. (bug 1266536, r=jimm) (84011349d1)
- Bug 1267253 - Delete gfxWindowsPlatform::RenderMode and replace it with a check against the default backend. r=bas (93cb6e503e)
- Introduce gfxConfig, a manager for graphics feature settings. (bug 1254899 part 3, r=milan) (97498ca46a)
- Bug 1262187: Allow D3D9 if D3D11 failed, behind the pref, but on by default. r=bas (8163e28b26)
- Bug 1178376 - Optionally fade in new progressively painted tiles r=nical (777bf1799d)
- Bug 1178376 - Allow progressive painting when low-precision tiles are disabled r=BenWa (3b8d84e19c)
- Bug 1178376 - Put progressive paint status in tile updates r=nical (935d3b46ed)
- Bug 1251778: Attempt to avoid presenting when the window is still resizing. r=jrmuizel (32b194a6f7)
- Hoist mWidget into the Compositor base class. (bug 1264545 part 1, r=nical) (aca26ec343)
- Lift compositor-accessed methods from nsIWidget into CompositorWidgetProxy. (bug 1264545 part 2, r=jimm) (609a23157a)
- Rename FeatureStatus::Crashed to CrashedInHandler. (bug 1254899 part 4, r=milan) (0eae23a3fd)
- Move DeviceInitData from gfxWindowsPlatform to gfxPlatforn. (bug 1254899 part 5, r=milan) (e31540ba18)
- Merge gfxWindowsPlatform::mAcceleration into gfxConfig. (bug 1254899 part 6, r=milan) (9d45cc8b87)
- Add another feature state for blacklisting and environment decisions. (bug 1254899 part 7, r=milan) (e7eee53cfb)
- Give FeatureState a public interface. (bug 1254899 part 8, r=milan) (52d8e7f355)
- Replace gfxWindowsPlatform::mD3D11Status with gfxConfig. (bug 1254899 part 9, r=jrmuizel) (90dc658b53)
- Replace gfxWindowsPlatform::mD2D1Status with gfxConfig. (bug 1254899 part 10, r=milan) (eb9474f309)
- Fix assertion failure in gfxConfig. (bug 1269565, r=milan) (0eb738ce66)
- Add gfxConfig to nsIGfxInfo, for about:support access. (bug 1254899 part 11, r=jrmuizel) (e770240152)
- Change Compositor::GetWidget to return a CompositorWidgetProxy. (bug 1264545 part 3, r=jimm) (fdf1d96255)
- Bug 1251778 - Followup: Remove unreferenced local variable. r=bustage (27579f5542)
- Use CompositorWidgetProxy in place of nsIWidget in the compositor. (bug 1264545 part 4, r=jimm) (80def1c2eb)
- Use CompositorWidgetProxy in place of nsIWidget in CompositorBridgeParent. (bug 1264545 part 5, r=jimm,kats) (67d0e1ef7d)
- Move CompositorWidgetProxy inheritance out of nsIWidget. (bug 1264545 part 6, r=jimm) (61075722c5)
- Bug 1251894 - In CompositorD3D11::CreateTexture, copy as much as the render target allows. r=bas (bf5fc6baa2)
- Bug 1266444: It is OK for us not to have texture sharing. r=jrmuizel (0b1885f89d)
- Bug 1266396 - Make TextureClient more robust against racy shutdown situations. r=Bas (b1d7f54643)
- Fix test bustage due to platform line-endings. Bug 1222624 (10b8cf3592)
- More test bustage from bug 1222624 (763c4c0bb9)
- Backed out 4 changesets (bug 1222624) to fix bug 1249572 (7ba3d433d0)
- Bug 1268230 - RunTime.cpp and ScriptLoader do not have to use MainThreadStopSyncLoopRunnable, r=khuey (88499a3982)
- Bug 1037725 - Add warning message in the console when worker spawn over limit. r=khuey (8af94dbc1d)
- Bug 1047663 - Disabling the cache in a tab should also disable it for all workers in that tab;r=khuey (5411d81682)
- Bug 1253793 Update ScriptLoader assertion to handle cancelation case. r=khuey (18c78d5651)
- Bug 1245768 - Implement a test for the correct error management when worker imports 3rd party scripts, r=bz (c1d3f290a9)
- Bug 1249673. Muted errors should be turned into NetworkError DOMExceptions when returning from importScripts on workers, instead of becoming NS_ERROR_FAILURE. r=baku (0358282cbe)
- Bug 1265405 - Add a dictionary to specify how PeriodicWave should be normalized (or not); r=smaug …normalized (or not); r=smaugu (201213146c)
- Bug 1251082. Restore comments in PageTransitionEvent.webidl that got lost when nsIDOMPageTransitionEvent.idl was migrated to webidl. r=bz The mentioned migration happened in http://hg.mozilla.org/mozilla-central/rev/e6377ca32f3d from bug 1031051. (2dfa309056)
- Bug 1266178 Make ServiceWorkerClient not assert if the document doesn't have an outer window. r=ehsan (eafb169c91)
- Bug 1259164 - Set ServiceWorkerMessageEvent.origin correctly when calling ServiceWorkerClient.postMessage(); r=bkelly (caeb65d10e)
- Bug 1246319 P1 Dedupe service worker registrar entries. r=baku (b76deef941)
- Bug 1246319 P2 Verify entries are deduped from the ServiceWorkerRegistrar. r=baku (8a4e348d6e)
- Bug 1246319 P3 Fix service worker registry value update. r=bz (14abf6b6ce)
- Bug 1247970 - Remove principal spec from service worker registrar file. r=baku (3c30130700)
- Bug 1249438 P1 Move guts of RegisterServiceWorker() into a protected method that can be tested in gtest. r=baku (488243196d)
- Bug 1249438 P2 Modify existing gtest to use RegisterServiceWorkerInternal. r=baku (e86c66891d)
- Bug 1249438 P3 Add a gtest that registers duplicate service worker registrations. r=baku (35e269f9af)
- Bug 1226443 P6 Ignore update() called during top level service worker script evaluation. r=ehsan (dcb9d02553)
- Bug 1241725 - about:serviceworkers "Active Cache Name" UUID should not contain null bytes, r=bkelly (4cddea6a67)
- Bug 1221852 - SharedWorker.port should be always not null, r=smaug (a9800274dc)
- Bug 1261428: Clean up a bit more. r=bz (4977e3d7a5)
- Bug 890284. Stop splitting textnodes in the XML content sink. r=peterv (a46dfca1cf)
- Bug 1211708 Allow themes to specify XBL bindings even in unprivileged documents r=sicking (82cf1a4023)
- Bug 915962 - Part 1: Allow pressing space to scroll the document if an editable element or form control is not focused; r=roc (cdb934af03)
- Bug 915962 - Part 2: Add a test case for pressing space when a tabindex=-1 and a button element is focused; r=roc (17dcf5cfd0)
- Bug 915962 - Part 3: Do not crash when pressing the space bar without having an element focused; r=roc (2161e62bc3)
- Bug 1180761, cancel the event earlier so that space doesn't trigger checkbox change and scroll, r=neil (2425cb76ad)
- Bug 1259182 - Shrink keyCodeData. r=bz. (737204af84)
- Bug 1193567 - Check result of ReadID in nsXBLPrototypeBinding::Read(). r=wchen (c9b1c35bf3)
- Bug 1173344 - Remove an intermediary root from nsXBLProtoImplField's FieldGetterImpl; r=jandem (5f42dd2e48)
- Bug 1207494 - Part 14: Remove use of expression closure from dom/xbl/. r=bz (21c7d3825f)
- align tests (fe34b613d3)
- Bug 1223702 - Fix some errors about wifi direct. r=hchang (568d86054a)
- Bug 1166274 - Part 1: Handle the callback and IPC message of setStaticIpMode correctly. r=vchang (8fb8d7f3b7)
- Bug 1133665 - [Flame][Wifi] The SSID that has set to be binded with MAC address is not hightlighted when user taps it. r=hchang (3165471d13)
- Bug 1207494 - Part 13: Remove use of expression closure from dom/wifi/. r=henry (dd9ad23a8a)
- Bug 1251856 - Disable U2F in all releases (fix for 1231681). r=baku (24ada10566)
- align tests (dae9ecd0ee)
- var-let (11a3cb0878)
- Bug 1184822 - Use classId to get provider. r=fabrice (1288eccd06)
- Bug 1247410 - "test for _nomap ids does not work correctly". r=dougt (f736a04f08)
- Bug 1035097 - Changed the type from 'radio' to 'radioType'. r=jdm (f9a0079152)
- Bug 1177871 - Add a timeout to XHR request of WifiGeoPositionProvider. r=jdm (2f6aa87c20)
- align code to 978593 with POST and location structure (d8ba75a759)
- Bug 1230685 - Replace function declarations with add_task statements in test_storage_value_array.js and test_unicode.js. r=mak (9822bf2215)
- Bug 1230683 - Replace try/catch with Assert.throws in test_storage_connection.js. r=mak (1c993fc37e)
- Bug 655722 - Rewrite _buildGUIDMap in the sync bookmark engine to use PlacesUtils.promiseBookmarksTree. r=mak (3795d26af0)
- Bug 1251057 - enable debug logging for rest.js requests and responses. r=adw (1062bcd113)
- Bug 503515 - Try and ensure exported certificates include an extension by default. r=keeler (505967ab7f)
- Bug 1017616 - Filter out some more unnecessary characters when exporting certs. r=keeler (e95838e362)
- Bug 1241614 - don't overflow:auto the container, use em to size the dialog to avoid hidpi visibility issues, r=dolske,ttaubert (8bd6c2b35b)
- Bug 1266851. Make <xmp> and <listing> use HTMLPreElement as their primary interface, per <whatwg/html#1015>.  r=peterv (6be7f9d6e9)
- Bug 1262184 - Block embed content loading when child of media element; r=bz (a297eeb378)
- Bug 1263696 - Block embed content loading when ancestor of object element with content; r=bz (89c143cbfe)
- Bug 1266077. Fix <base> href getter to follow the spec; it should be using the fallback base URI to resolve against, not the document URI. r=bkelly (e757b23a14)
- Bug 1168079 nsTextEditRules::CollapseSelectionToTrailingBRIfNeeded() should ensure that there is a selection before calling nsEditor::GetStartNodeAndOffset() r=ehsan (6c283bf3a7)
- Bug 898321 - Return success from nsTableEditor::GetCellAt if frame not found; r=ehsan (0d09143b95)
- Bug 387687 - wrap quotes in plain text replies to window. r=masayuki (ca51437018)
- Bug 1247483 - Only replace nodes in nsHTMLEditor::ReplaceOrphanedStructure if all nodes in node list are descendants of replacement node. r=ehsan (8416037da2)
- bug 1266496 - fire some selection events for proxied accessibles r=davidb (8806de7dd9)
- bug 1266518 - add a new event message for AccSelChangeEvents r=davidb (46af183cab)
- Bug 953265: Adjust Opus bitrate in WebRTC to pass >8KHz audio, and comment r=bwc (b0be6a326e)
- Bug 1221473: Do not treat answer as authoritative wrt payload types. r=drno (d27409209e)
- Bug 1241321 - No RTCP stats for audio streams. r=rjesup (ec0222694e)
- bug 1250492 - use tl::Max instead of std::max to get rid of a static constructor r=jesup (3cebbc8969)
- Bug 1254187: Fix maxBitrate to respect simulcast. r=jesup (e569e54b57)
- Bug 1158931 - Fix static assertion compilation error; r=snorp (eb27881746)
- No bug, fix WebrtcMediaCodecVP8VideoCodec.cpp warnings (a983544581)
- Bug 1252737 - use size_t instead of uint32_t for InitEncode(). r=jesup (57c3abc9fa)
- Bug 1208371 - Never send more than one disabled frame in a row to the WebRTC encoder. r=jesup (ec0c28822b)
- Bug 1208371 - Do image format conversion async in MediaPipeline. r=jesup (032efec783)
- Bug 1266685 - Don't pass too many frames to the MediaPipelineTransmit VideoFrameConverter. r=jesup (21774a8d25)
- Bug 1266644 - Rename StreamBuffer to StreamTracks. r=jesup r=pehrsons (21906fe1f7)
- Bug 1208371 - Don't treat audio chunks as mutable in MediaPipeline. r=padenot (3878ef4332)
- Bug 1246310 - Let MediaPipelineReceive tracks start at 0. r=jesup (b468ff8d48)
- Bug 1266644 - Rename DOMMediaStream:: CreateXXXStream to DOMMediaStream:: CreateXXXStreamAsInput. r=jesup r=pehrsons (fe4b6d70bc)
- Bug 1234578: Add an assertion. r=drno, a=abillings (f1a2c8d841)
- bug 1250492 - include sstream in SdpMediaSection.h instead of iostream r=jesup (110b5c2eca)
- Bug 1264470 - a=identity is a long attribute, r=bwc (5848194fe9)
- Bug 1256750: Remove unnecessary sscanf_s parameter on windows, and fix format string everywhere else. r=jesup (371c0db476)
- Bug 1204082 - try strtoull instead. r=mt (a0313aa87c)
- Bug 1113443 - reject each media type with approriate default. r=bwc (a72ff312d1)
- Bug 1095793 - use mid if provided to place candidate in msection. r=bwc (2c29b21fac)
- Bug 1252699 - Set WEBRTC_DETECT_ARM_NEON when optional neon is requested. r=jesup (722e2043a5)
- Bug 1229475 - webrtc: Call opus tonality_analysis_init. r=jesup (1cf8cc2cd7)
- Bug 1254876: assert windows recording is shut down r=pkerr (1f2cb69073)
- Bug 1227481 - added a memset on aec. r=jesup (532026ce20)
- Bug 1254507 - Fix leak in WebRTC DesktopApplication class. r=jesup (54da72aeb4)
- Bug 1196542 - share only windows with non-zero area. r=pkerr (94595ec463)
- Bug 1202087 - Filter out non-shareable application for win8 or greater. r=jesup (d989956802)
- Bug 1216529 - WebRTC: Request camera permission before accessing camera APIs. r=gcp (24b6699226)
- Bug 1237630 - Part 1: Video freeze from WebRTC sender. r=rjesup (02daa8b5b7)
- Bug 1237630 - Part 2: remove LOG statement generating a now defunct error condition. r=rjesup (c6002ef12f)
- Bug 1248335: avoid using SvcInternal structure entirely, as system-vpx may not have it r=pkerr (ef9b21f20c)
- Bug 1234571: unregister encoded-frame callback when releasing codec databases r=pkerr (321bd5166b)
- Bug 820972 - Comment out colorTable[] because we don't need it. r=jesup. (60b10803d5)
- cleanup and missing test stuff (e2be0331d9)
- Merge remote-tracking branch 'upstream/dev' into winbuild (3a3bb0b315)
- layout/media: fix export symbol list, fix build bustage (f9f5bfe14c)
2024-09-10 22:30:49 +08:00

1519 lines
48 KiB
C++

/* -*- Mode: C++; tab-width: 2; indent-tabs-mode: nil; c-basic-offset: 2 -*-*/
/* This Source Code Form is subject to the terms of the Mozilla Public
* License, v. 2.0. If a copy of the MPL was not distributed with this file,
* You can obtain one at http://mozilla.org/MPL/2.0/. */
#include "DOMMediaStream.h"
#include "nsContentUtils.h"
#include "nsServiceManagerUtils.h"
#include "nsIScriptError.h"
#include "nsIUUIDGenerator.h"
#include "nsPIDOMWindow.h"
#include "mozilla/dom/MediaStreamBinding.h"
#include "mozilla/dom/LocalMediaStreamBinding.h"
#include "mozilla/dom/AudioNode.h"
#include "AudioChannelAgent.h"
#include "mozilla/dom/AudioTrack.h"
#include "mozilla/dom/AudioTrackList.h"
#include "mozilla/dom/VideoTrack.h"
#include "mozilla/dom/VideoTrackList.h"
#include "mozilla/dom/HTMLCanvasElement.h"
#include "mozilla/media/MediaUtils.h"
#include "MediaStreamGraph.h"
#include "AudioStreamTrack.h"
#include "VideoStreamTrack.h"
#include "Layers.h"
// GetCurrentTime is defined in winbase.h as zero argument macro forwarding to
// GetTickCount() and conflicts with NS_DECL_NSIDOMMEDIASTREAM, containing
// currentTime getter.
#ifdef GetCurrentTime
#undef GetCurrentTime
#endif
#ifdef LOG
#undef LOG
#endif
// GetCurrentTime is defined in winbase.h as zero argument macro forwarding to
// GetTickCount() and conflicts with MediaStream::GetCurrentTime.
#ifdef GetCurrentTime
#undef GetCurrentTime
#endif
using namespace mozilla;
using namespace mozilla::dom;
using namespace mozilla::layers;
using namespace mozilla::media;
static LazyLogModule gMediaStreamLog("MediaStream");
#define LOG(type, msg) MOZ_LOG(gMediaStreamLog, type, msg)
const TrackID TRACK_VIDEO_PRIMARY = 1;
DOMMediaStream::TrackPort::TrackPort(MediaInputPort* aInputPort,
MediaStreamTrack* aTrack,
const InputPortOwnership aOwnership)
: mInputPort(aInputPort)
, mTrack(aTrack)
, mOwnership(aOwnership)
{
// XXX Bug 1124630. nsDOMCameraControl requires adding a track without and
// input port.
// MOZ_ASSERT(mInputPort);
MOZ_ASSERT(mTrack);
MOZ_COUNT_CTOR(TrackPort);
}
DOMMediaStream::TrackPort::~TrackPort()
{
MOZ_COUNT_DTOR(TrackPort);
if (mOwnership == InputPortOwnership::OWNED && mInputPort) {
mInputPort->Destroy();
mInputPort = nullptr;
}
}
void
DOMMediaStream::TrackPort::DestroyInputPort()
{
if (mInputPort) {
mInputPort->Destroy();
mInputPort = nullptr;
}
}
MediaStream*
DOMMediaStream::TrackPort::GetSource() const
{
return mInputPort ? mInputPort->GetSource() : nullptr;
}
TrackID
DOMMediaStream::TrackPort::GetSourceTrackId() const
{
return mInputPort ? mInputPort->GetSourceTrackId() : TRACK_INVALID;
}
already_AddRefed<Pledge<bool>>
DOMMediaStream::TrackPort::BlockSourceTrackId(TrackID aTrackId)
{
if (mInputPort) {
return mInputPort->BlockSourceTrackId(aTrackId);
}
RefPtr<Pledge<bool>> rejected = new Pledge<bool>();
rejected->Reject(NS_ERROR_FAILURE);
return rejected.forget();
}
NS_IMPL_CYCLE_COLLECTION(DOMMediaStream::TrackPort, mTrack)
NS_IMPL_CYCLE_COLLECTION_ROOT_NATIVE(DOMMediaStream::TrackPort, AddRef)
NS_IMPL_CYCLE_COLLECTION_UNROOT_NATIVE(DOMMediaStream::TrackPort, Release)
NS_IMPL_CYCLE_COLLECTING_ADDREF(MediaStreamTrackSourceGetter)
NS_IMPL_CYCLE_COLLECTING_RELEASE(MediaStreamTrackSourceGetter)
NS_INTERFACE_MAP_BEGIN_CYCLE_COLLECTION(MediaStreamTrackSourceGetter)
NS_INTERFACE_MAP_ENTRY(nsISupports)
NS_INTERFACE_MAP_END
NS_IMPL_CYCLE_COLLECTION_0(MediaStreamTrackSourceGetter)
/**
* Listener registered on the Owned stream to detect added and ended owned
* tracks for keeping the list of MediaStreamTracks in sync with the tracks
* added and ended directly at the source.
*/
class DOMMediaStream::OwnedStreamListener : public MediaStreamListener {
public:
explicit OwnedStreamListener(DOMMediaStream* aStream)
: mStream(aStream)
{}
void Forget() { mStream = nullptr; }
void DoNotifyTrackCreated(TrackID aTrackID, MediaSegment::Type aType,
MediaStream* aInputStream, TrackID aInputTrackID)
{
MOZ_ASSERT(NS_IsMainThread());
if (!mStream) {
return;
}
MediaStreamTrack* track =
mStream->FindOwnedDOMTrack(aInputStream, aInputTrackID);
if (!track) {
// Track had not been created on main thread before, create it now.
NS_WARN_IF_FALSE(!mStream->mTracks.IsEmpty(),
"A new track was detected on the input stream; creating "
"a corresponding MediaStreamTrack. Initial tracks "
"should be added manually to immediately and "
"synchronously be available to JS.");
RefPtr<MediaStreamTrackSource> source;
if (mStream->mTrackSourceGetter) {
source = mStream->mTrackSourceGetter->GetMediaStreamTrackSource(aTrackID);
}
if (!source) {
NS_ASSERTION(false, "Dynamic track created without an explicit TrackSource");
nsPIDOMWindow* window = mStream->GetParentObject();
nsIDocument* doc = window ? window->GetExtantDoc() : nullptr;
nsIPrincipal* principal = doc ? doc->NodePrincipal() : nullptr;
source = new BasicUnstoppableTrackSource(principal);
}
track = mStream->CreateDOMTrack(aTrackID, aType, source);
}
}
void DoNotifyTrackEnded(MediaStream* aInputStream, TrackID aInputTrackID)
{
MOZ_ASSERT(NS_IsMainThread());
if (!mStream) {
return;
}
RefPtr<MediaStreamTrack> track =
mStream->FindOwnedDOMTrack(aInputStream, aInputTrackID);
NS_ASSERTION(track, "Owned MediaStreamTracks must be known by the DOMMediaStream");
if (track) {
LOG(LogLevel::Debug, ("DOMMediaStream %p MediaStreamTrack %p ended at the source. Marking it ended.",
mStream, track.get()));
track->NotifyEnded();
}
}
void NotifyQueuedTrackChanges(MediaStreamGraph* aGraph, TrackID aID,
StreamTime aTrackOffset, uint32_t aTrackEvents,
const MediaSegment& aQueuedMedia,
MediaStream* aInputStream,
TrackID aInputTrackID) override
{
if (aTrackEvents & TRACK_EVENT_CREATED) {
nsCOMPtr<nsIRunnable> runnable =
NS_NewRunnableMethodWithArgs<TrackID, MediaSegment::Type, MediaStream*, TrackID>(
this, &OwnedStreamListener::DoNotifyTrackCreated,
aID, aQueuedMedia.GetType(), aInputStream, aInputTrackID);
aGraph->DispatchToMainThreadAfterStreamStateUpdate(runnable.forget());
} else if (aTrackEvents & TRACK_EVENT_ENDED) {
nsCOMPtr<nsIRunnable> runnable =
NS_NewRunnableMethodWithArgs<MediaStream*, TrackID>(
this, &OwnedStreamListener::DoNotifyTrackEnded,
aInputStream, aInputTrackID);
aGraph->DispatchToMainThreadAfterStreamStateUpdate(runnable.forget());
}
}
private:
// These fields may only be accessed on the main thread
DOMMediaStream* mStream;
};
/**
* Listener registered on the Playback stream to detect when tracks end and when
* all new tracks this iteration have been created - for when several tracks are
* queued by the source and committed all at once.
*/
class DOMMediaStream::PlaybackStreamListener : public MediaStreamListener {
public:
explicit PlaybackStreamListener(DOMMediaStream* aStream)
: mStream(aStream)
{}
void Forget()
{
MOZ_ASSERT(NS_IsMainThread());
mStream = nullptr;
}
void DoNotifyTrackEnded(MediaStream* aInputStream,
TrackID aInputTrackID)
{
MOZ_ASSERT(NS_IsMainThread());
if (!mStream) {
return;
}
LOG(LogLevel::Debug, ("DOMMediaStream %p Track %u of stream %p ended",
mStream, aInputTrackID, aInputStream));
RefPtr<MediaStreamTrack> track =
mStream->FindPlaybackDOMTrack(aInputStream, aInputTrackID);
if (!track) {
LOG(LogLevel::Debug, ("DOMMediaStream %p Not a playback track.", mStream));
return;
}
LOG(LogLevel::Debug, ("DOMMediaStream %p Playback track; notifying stream listeners.",
mStream));
mStream->NotifyTrackRemoved(track);
RefPtr<TrackPort> endedPort = mStream->FindPlaybackTrackPort(*track);
NS_ASSERTION(endedPort, "Playback track should have a TrackPort");
if (endedPort && IsTrackIDExplicit(endedPort->GetSourceTrackId())) {
// If a track connected to a locked-track input port ends, we destroy the
// port to allow our playback stream to finish.
// XXX (bug 1208316) This should not be necessary when MediaStreams don't
// finish but instead become inactive.
endedPort->DestroyInputPort();
}
}
void DoNotifyFinishedTrackCreation()
{
MOZ_ASSERT(NS_IsMainThread());
if (!mStream) {
return;
}
mStream->NotifyTracksCreated();
}
// The methods below are called on the MediaStreamGraph thread.
void NotifyQueuedTrackChanges(MediaStreamGraph* aGraph, TrackID aID,
StreamTime aTrackOffset, uint32_t aTrackEvents,
const MediaSegment& aQueuedMedia,
MediaStream* aInputStream,
TrackID aInputTrackID) override
{
if (aTrackEvents & TRACK_EVENT_ENDED) {
nsCOMPtr<nsIRunnable> runnable =
NS_NewRunnableMethodWithArgs<StorensRefPtrPassByPtr<MediaStream>, TrackID>(
this, &PlaybackStreamListener::DoNotifyTrackEnded, aInputStream, aInputTrackID);
aGraph->DispatchToMainThreadAfterStreamStateUpdate(runnable.forget());
}
}
void NotifyFinishedTrackCreation(MediaStreamGraph* aGraph) override
{
nsCOMPtr<nsIRunnable> runnable =
NS_NewRunnableMethod(this, &PlaybackStreamListener::DoNotifyFinishedTrackCreation);
aGraph->DispatchToMainThreadAfterStreamStateUpdate(runnable.forget());
}
private:
// These fields may only be accessed on the main thread
DOMMediaStream* mStream;
};
NS_IMPL_CYCLE_COLLECTION_CLASS(DOMMediaStream)
NS_IMPL_CYCLE_COLLECTION_UNLINK_BEGIN_INHERITED(DOMMediaStream,
DOMEventTargetHelper)
tmp->Destroy();
NS_IMPL_CYCLE_COLLECTION_UNLINK(mWindow)
NS_IMPL_CYCLE_COLLECTION_UNLINK(mOwnedTracks)
NS_IMPL_CYCLE_COLLECTION_UNLINK(mTracks)
NS_IMPL_CYCLE_COLLECTION_UNLINK(mConsumersToKeepAlive)
NS_IMPL_CYCLE_COLLECTION_UNLINK(mTrackSourceGetter)
NS_IMPL_CYCLE_COLLECTION_UNLINK(mPrincipal)
NS_IMPL_CYCLE_COLLECTION_UNLINK(mVideoPrincipal)
NS_IMPL_CYCLE_COLLECTION_UNLINK_END
NS_IMPL_CYCLE_COLLECTION_TRAVERSE_BEGIN_INHERITED(DOMMediaStream,
DOMEventTargetHelper)
NS_IMPL_CYCLE_COLLECTION_TRAVERSE(mWindow)
NS_IMPL_CYCLE_COLLECTION_TRAVERSE(mOwnedTracks)
NS_IMPL_CYCLE_COLLECTION_TRAVERSE(mTracks)
NS_IMPL_CYCLE_COLLECTION_TRAVERSE(mConsumersToKeepAlive)
NS_IMPL_CYCLE_COLLECTION_TRAVERSE(mTrackSourceGetter)
NS_IMPL_CYCLE_COLLECTION_TRAVERSE(mPrincipal)
NS_IMPL_CYCLE_COLLECTION_TRAVERSE(mVideoPrincipal)
NS_IMPL_CYCLE_COLLECTION_TRAVERSE_END
NS_IMPL_ADDREF_INHERITED(DOMMediaStream, DOMEventTargetHelper)
NS_IMPL_RELEASE_INHERITED(DOMMediaStream, DOMEventTargetHelper)
NS_INTERFACE_MAP_BEGIN_CYCLE_COLLECTION_INHERITED(DOMMediaStream)
NS_INTERFACE_MAP_ENTRY(DOMMediaStream)
NS_INTERFACE_MAP_END_INHERITING(DOMEventTargetHelper)
NS_IMPL_ADDREF_INHERITED(DOMLocalMediaStream, DOMMediaStream)
NS_IMPL_RELEASE_INHERITED(DOMLocalMediaStream, DOMMediaStream)
NS_INTERFACE_MAP_BEGIN(DOMLocalMediaStream)
NS_INTERFACE_MAP_ENTRY(DOMLocalMediaStream)
NS_INTERFACE_MAP_END_INHERITING(DOMMediaStream)
NS_IMPL_CYCLE_COLLECTION_INHERITED(DOMAudioNodeMediaStream, DOMMediaStream,
mStreamNode)
NS_IMPL_ADDREF_INHERITED(DOMAudioNodeMediaStream, DOMMediaStream)
NS_IMPL_RELEASE_INHERITED(DOMAudioNodeMediaStream, DOMMediaStream)
NS_INTERFACE_MAP_BEGIN_CYCLE_COLLECTION_INHERITED(DOMAudioNodeMediaStream)
NS_INTERFACE_MAP_END_INHERITING(DOMMediaStream)
DOMMediaStream::DOMMediaStream(nsPIDOMWindow* aWindow,
MediaStreamTrackSourceGetter* aTrackSourceGetter)
: mLogicalStreamStartTime(0), mWindow(aWindow),
mInputStream(nullptr), mOwnedStream(nullptr), mPlaybackStream(nullptr),
mTracksPendingRemoval(0), mTrackSourceGetter(aTrackSourceGetter),
mTracksCreated(false), mNotifiedOfMediaStreamGraphShutdown(false)
{
nsresult rv;
nsCOMPtr<nsIUUIDGenerator> uuidgen =
do_GetService("@mozilla.org/uuid-generator;1", &rv);
if (NS_SUCCEEDED(rv) && uuidgen) {
nsID uuid;
memset(&uuid, 0, sizeof(uuid));
rv = uuidgen->GenerateUUIDInPlace(&uuid);
if (NS_SUCCEEDED(rv)) {
char buffer[NSID_LENGTH];
uuid.ToProvidedString(buffer);
mID = NS_ConvertASCIItoUTF16(buffer);
}
}
}
DOMMediaStream::~DOMMediaStream()
{
Destroy();
}
void
DOMMediaStream::Destroy()
{
LOG(LogLevel::Debug, ("DOMMediaStream %p Being destroyed.", this));
if (mOwnedListener) {
mOwnedListener->Forget();
mOwnedListener = nullptr;
}
if (mPlaybackListener) {
mPlaybackListener->Forget();
mPlaybackListener = nullptr;
}
for (const RefPtr<TrackPort>& info : mTracks) {
// We must remove ourselves from each track's principal change observer list
// before we die. CC may have cleared info->mTrack so guard against it.
if (info->GetTrack()) {
info->GetTrack()->RemovePrincipalChangeObserver(this);
}
}
if (mPlaybackPort) {
mPlaybackPort->Destroy();
mPlaybackPort = nullptr;
}
if (mOwnedPort) {
mOwnedPort->Destroy();
mOwnedPort = nullptr;
}
if (mPlaybackStream) {
mPlaybackStream->UnregisterUser();
mPlaybackStream = nullptr;
}
if (mOwnedStream) {
mOwnedStream->UnregisterUser();
mOwnedStream = nullptr;
}
if (mInputStream) {
mInputStream->UnregisterUser();
mInputStream = nullptr;
}
}
JSObject*
DOMMediaStream::WrapObject(JSContext* aCx, JS::Handle<JSObject*> aGivenProto)
{
return dom::MediaStreamBinding::Wrap(aCx, this, aGivenProto);
}
/* static */ already_AddRefed<DOMMediaStream>
DOMMediaStream::Constructor(const GlobalObject& aGlobal,
ErrorResult& aRv)
{
Sequence<OwningNonNull<MediaStreamTrack>> emptyTrackSeq;
return Constructor(aGlobal, emptyTrackSeq, aRv);
}
/* static */ already_AddRefed<DOMMediaStream>
DOMMediaStream::Constructor(const GlobalObject& aGlobal,
const DOMMediaStream& aStream,
ErrorResult& aRv)
{
nsTArray<RefPtr<MediaStreamTrack>> tracks;
aStream.GetTracks(tracks);
Sequence<OwningNonNull<MediaStreamTrack>> nonNullTrackSeq;
if (!nonNullTrackSeq.SetLength(tracks.Length(), fallible)) {
MOZ_ASSERT(false);
aRv.Throw(NS_ERROR_OUT_OF_MEMORY);
return nullptr;
}
for (size_t i = 0; i < tracks.Length(); ++i) {
nonNullTrackSeq[i] = tracks[i];
}
return Constructor(aGlobal, nonNullTrackSeq, aRv);
}
/* static */ already_AddRefed<DOMMediaStream>
DOMMediaStream::Constructor(const GlobalObject& aGlobal,
const Sequence<OwningNonNull<MediaStreamTrack>>& aTracks,
ErrorResult& aRv)
{
nsCOMPtr<nsPIDOMWindow> ownerWindow = do_QueryInterface(aGlobal.GetAsSupports());
if (!ownerWindow) {
aRv.Throw(NS_ERROR_FAILURE);
return nullptr;
}
// Streams created from JS cannot have dynamically created tracks.
MediaStreamTrackSourceGetter* getter = nullptr;
RefPtr<DOMMediaStream> newStream = new DOMMediaStream(ownerWindow, getter);
for (MediaStreamTrack& track : aTracks) {
if (!newStream->GetPlaybackStream()) {
MOZ_RELEASE_ASSERT(track.Graph());
newStream->InitPlaybackStreamCommon(track.Graph());
}
newStream->AddTrack(track);
}
if (!newStream->GetPlaybackStream()) {
MOZ_ASSERT(aTracks.IsEmpty());
MediaStreamGraph* graph =
MediaStreamGraph::GetInstance(MediaStreamGraph::SYSTEM_THREAD_DRIVER,
AudioChannel::Normal);
newStream->InitPlaybackStreamCommon(graph);
}
return newStream.forget();
}
double
DOMMediaStream::CurrentTime()
{
if (!mPlaybackStream) {
return 0.0;
}
return mPlaybackStream->
StreamTimeToSeconds(mPlaybackStream->GetCurrentTime() - mLogicalStreamStartTime);
}
void
DOMMediaStream::GetId(nsAString& aID) const
{
aID = mID;
}
void
DOMMediaStream::GetAudioTracks(nsTArray<RefPtr<AudioStreamTrack> >& aTracks) const
{
for (const RefPtr<TrackPort>& info : mTracks) {
AudioStreamTrack* t = info->GetTrack()->AsAudioStreamTrack();
if (t) {
aTracks.AppendElement(t);
}
}
}
void
DOMMediaStream::GetVideoTracks(nsTArray<RefPtr<VideoStreamTrack> >& aTracks) const
{
for (const RefPtr<TrackPort>& info : mTracks) {
VideoStreamTrack* t = info->GetTrack()->AsVideoStreamTrack();
if (t) {
aTracks.AppendElement(t);
}
}
}
void
DOMMediaStream::GetTracks(nsTArray<RefPtr<MediaStreamTrack> >& aTracks) const
{
for (const RefPtr<TrackPort>& info : mTracks) {
aTracks.AppendElement(info->GetTrack());
}
}
void
DOMMediaStream::AddTrack(MediaStreamTrack& aTrack)
{
MOZ_RELEASE_ASSERT(mPlaybackStream);
RefPtr<ProcessedMediaStream> dest = mPlaybackStream->AsProcessedStream();
MOZ_ASSERT(dest);
if (!dest) {
return;
}
LOG(LogLevel::Info, ("DOMMediaStream %p Adding track %p (from stream %p with ID %d)",
this, &aTrack, aTrack.mOwningStream.get(), aTrack.mTrackID));
if (mPlaybackStream->Graph() != aTrack.Graph()) {
NS_ASSERTION(false, "Cannot combine tracks from different MediaStreamGraphs");
LOG(LogLevel::Error, ("DOMMediaStream %p Own MSG %p != aTrack's MSG %p",
this, mPlaybackStream->Graph(), aTrack.Graph()));
nsAutoString trackId;
aTrack.GetId(trackId);
const char16_t* params[] = { trackId.get() };
nsCOMPtr<nsPIDOMWindow> pWindow = do_QueryInterface(GetParentObject());
nsIDocument* document = pWindow ? pWindow->GetExtantDoc() : nullptr;
nsContentUtils::ReportToConsole(nsIScriptError::errorFlag,
NS_LITERAL_CSTRING("Media"),
document,
nsContentUtils::eDOM_PROPERTIES,
"MediaStreamAddTrackDifferentAudioChannel",
params, ArrayLength(params));
return;
}
if (HasTrack(aTrack)) {
LOG(LogLevel::Debug, ("DOMMediaStream %p already contains track %p", this, &aTrack));
return;
}
// Hook up the underlying track with our underlying playback stream.
RefPtr<MediaInputPort> inputPort =
GetPlaybackStream()->AllocateInputPort(aTrack.GetOwnedStream(),
aTrack.mTrackID);
RefPtr<TrackPort> trackPort =
new TrackPort(inputPort, &aTrack, TrackPort::InputPortOwnership::OWNED);
mTracks.AppendElement(trackPort.forget());
NotifyTrackAdded(&aTrack);
LOG(LogLevel::Debug, ("DOMMediaStream %p Added track %p", this, &aTrack));
}
void
DOMMediaStream::RemoveTrack(MediaStreamTrack& aTrack)
{
LOG(LogLevel::Info, ("DOMMediaStream %p Removing track %p (from stream %p with ID %d)",
this, &aTrack, aTrack.mOwningStream.get(), aTrack.mTrackID));
RefPtr<TrackPort> toRemove = FindPlaybackTrackPort(aTrack);
if (!toRemove) {
LOG(LogLevel::Debug, ("DOMMediaStream %p does not contain track %p", this, &aTrack));
return;
}
// If the track comes from a TRACK_ANY input port (i.e., mOwnedPort), we need
// to block it in the port. Doing this for a locked track is still OK as it
// will first block the track, then destroy the port. Both cause the track to
// end.
BlockPlaybackTrack(toRemove);
DebugOnly<bool> removed = mTracks.RemoveElement(toRemove);
MOZ_ASSERT(removed);
NotifyTrackRemoved(&aTrack);
LOG(LogLevel::Debug, ("DOMMediaStream %p Removed track %p", this, &aTrack));
}
class ClonedStreamSourceGetter :
public MediaStreamTrackSourceGetter
{
public:
NS_DECL_ISUPPORTS_INHERITED
NS_DECL_CYCLE_COLLECTION_CLASS_INHERITED(ClonedStreamSourceGetter,
MediaStreamTrackSourceGetter)
explicit ClonedStreamSourceGetter(DOMMediaStream* aStream)
: mStream(aStream) {}
already_AddRefed<MediaStreamTrackSource>
GetMediaStreamTrackSource(TrackID aInputTrackID) override
{
MediaStreamTrack* sourceTrack =
mStream->FindOwnedDOMTrack(mStream->GetOwnedStream(), aInputTrackID);
MOZ_RELEASE_ASSERT(sourceTrack);
return do_AddRef(&sourceTrack->GetSource());
}
protected:
virtual ~ClonedStreamSourceGetter() {}
RefPtr<DOMMediaStream> mStream;
};
NS_IMPL_ADDREF_INHERITED(ClonedStreamSourceGetter,
MediaStreamTrackSourceGetter)
NS_IMPL_RELEASE_INHERITED(ClonedStreamSourceGetter,
MediaStreamTrackSourceGetter)
NS_INTERFACE_MAP_BEGIN_CYCLE_COLLECTION_INHERITED(ClonedStreamSourceGetter)
NS_INTERFACE_MAP_END_INHERITING(MediaStreamTrackSourceGetter)
NS_IMPL_CYCLE_COLLECTION_INHERITED(ClonedStreamSourceGetter,
MediaStreamTrackSourceGetter,
mStream)
already_AddRefed<DOMMediaStream>
DOMMediaStream::Clone()
{
return CloneInternal(TrackForwardingOption::CURRENT);
}
already_AddRefed<DOMMediaStream>
DOMMediaStream::CloneInternal(TrackForwardingOption aForwarding)
{
RefPtr<DOMMediaStream> newStream =
new DOMMediaStream(GetParentObject(), new ClonedStreamSourceGetter(this));
LOG(LogLevel::Info, ("DOMMediaStream %p created clone %p, forwarding %s tracks",
this, newStream.get(),
aForwarding == TrackForwardingOption::ALL
? "all" : "current"));
MOZ_RELEASE_ASSERT(mPlaybackStream);
MOZ_RELEASE_ASSERT(mPlaybackStream->Graph());
MediaStreamGraph* graph = mPlaybackStream->Graph();
// We initiate the owned and playback streams first, since we need to create
// all existing DOM tracks before we add the generic input port from
// mInputStream to mOwnedStream (see AllocateInputPort wrt. destination
// TrackID as to why).
newStream->InitOwnedStreamCommon(graph);
newStream->InitPlaybackStreamCommon(graph);
// Set up existing DOM tracks.
TrackID allocatedTrackID = 1;
for (const RefPtr<TrackPort>& info : mTracks) {
MediaStreamTrack& track = *info->GetTrack();
LOG(LogLevel::Debug, ("DOMMediaStream %p forwarding external track %p to clone %p",
this, &track, newStream.get()));
RefPtr<MediaStreamTrack> trackClone =
newStream->CloneDOMTrack(track, allocatedTrackID++);
}
if (aForwarding == TrackForwardingOption::ALL) {
// Set up an input port from our input stream to the new DOM stream's owned
// stream, to allow for dynamically added tracks at the source to appear in
// the clone. The clone may treat mInputStream as its own mInputStream but
// ownership remains with us.
newStream->mInputStream = mInputStream;
if (mInputStream) {
// We have already set up track-locked input ports for all existing DOM
// tracks, so now we need to block those in the generic input port to
// avoid ending up with double instances of them.
nsTArray<TrackID> tracksToBlock;
for (const RefPtr<TrackPort>& info : mOwnedTracks) {
tracksToBlock.AppendElement(info->GetTrack()->mTrackID);
}
newStream->mInputStream->RegisterUser();
newStream->mOwnedPort =
newStream->mOwnedStream->AllocateInputPort(mInputStream,
TRACK_ANY, TRACK_ANY, 0, 0,
&tracksToBlock);
}
}
return newStream.forget();
}
MediaStreamTrack*
DOMMediaStream::GetTrackById(const nsString& aId)
{
for (const RefPtr<TrackPort>& info : mTracks) {
nsString id;
info->GetTrack()->GetId(id);
if (id == aId) {
return info->GetTrack();
}
}
return nullptr;
}
MediaStreamTrack*
DOMMediaStream::GetOwnedTrackById(const nsString& aId)
{
for (const RefPtr<TrackPort>& info : mOwnedTracks) {
nsString id;
info->GetTrack()->GetId(id);
if (id == aId) {
return info->GetTrack();
}
}
return nullptr;
}
bool
DOMMediaStream::HasTrack(const MediaStreamTrack& aTrack) const
{
return !!FindPlaybackTrackPort(aTrack);
}
bool
DOMMediaStream::OwnsTrack(const MediaStreamTrack& aTrack) const
{
return !!FindOwnedTrackPort(aTrack);
}
bool
DOMMediaStream::AddDirectListener(MediaStreamDirectListener* aListener)
{
if (GetInputStream() && GetInputStream()->AsSourceStream()) {
GetInputStream()->AsSourceStream()->AddDirectListener(aListener);
return true; // application should ignore NotifyQueuedTrackData
}
return false;
}
void
DOMMediaStream::RemoveDirectListener(MediaStreamDirectListener* aListener)
{
if (GetInputStream() && GetInputStream()->AsSourceStream()) {
GetInputStream()->AsSourceStream()->RemoveDirectListener(aListener);
}
}
bool
DOMMediaStream::IsFinished()
{
return !mPlaybackStream || mPlaybackStream->IsFinished();
}
void
DOMMediaStream::InitSourceStream(MediaStreamGraph* aGraph)
{
InitInputStreamCommon(aGraph->CreateSourceStream(nullptr), aGraph);
InitOwnedStreamCommon(aGraph);
InitPlaybackStreamCommon(aGraph);
}
void
DOMMediaStream::InitTrackUnionStream(MediaStreamGraph* aGraph)
{
InitInputStreamCommon(aGraph->CreateTrackUnionStream(nullptr), aGraph);
InitOwnedStreamCommon(aGraph);
InitPlaybackStreamCommon(aGraph);
}
void
DOMMediaStream::InitAudioCaptureStream(nsIPrincipal* aPrincipal, MediaStreamGraph* aGraph)
{
const TrackID AUDIO_TRACK = 1;
RefPtr<BasicUnstoppableTrackSource> audioCaptureSource =
new BasicUnstoppableTrackSource(aPrincipal, MediaSourceEnum::AudioCapture);
AudioCaptureStream* audioCaptureStream =
static_cast<AudioCaptureStream*>(aGraph->CreateAudioCaptureStream(this, AUDIO_TRACK));
InitInputStreamCommon(audioCaptureStream, aGraph);
InitOwnedStreamCommon(aGraph);
InitPlaybackStreamCommon(aGraph);
CreateDOMTrack(AUDIO_TRACK, MediaSegment::AUDIO, audioCaptureSource);
audioCaptureStream->Start();
}
void
DOMMediaStream::InitInputStreamCommon(MediaStream* aStream,
MediaStreamGraph* aGraph)
{
MOZ_ASSERT(!mOwnedStream, "Input stream must be initialized before owned stream");
mInputStream = aStream;
mInputStream->RegisterUser();
}
void
DOMMediaStream::InitOwnedStreamCommon(MediaStreamGraph* aGraph)
{
MOZ_ASSERT(!mPlaybackStream, "Owned stream must be initialized before playback stream");
// We pass null as the wrapper since it is only used to signal finished
// streams. This is only needed for the playback stream.
mOwnedStream = aGraph->CreateTrackUnionStream(nullptr);
mOwnedStream->SetAutofinish(true);
mOwnedStream->RegisterUser();
if (mInputStream) {
mOwnedPort = mOwnedStream->AllocateInputPort(mInputStream);
}
// Setup track listeners
mOwnedListener = new OwnedStreamListener(this);
mOwnedStream->AddListener(mOwnedListener);
}
void
DOMMediaStream::InitPlaybackStreamCommon(MediaStreamGraph* aGraph)
{
mPlaybackStream = aGraph->CreateTrackUnionStream(this);
mPlaybackStream->SetAutofinish(true);
mPlaybackStream->RegisterUser();
if (mOwnedStream) {
mPlaybackPort = mPlaybackStream->AllocateInputPort(mOwnedStream);
}
mPlaybackListener = new PlaybackStreamListener(this);
mPlaybackStream->AddListener(mPlaybackListener);
LOG(LogLevel::Debug, ("DOMMediaStream %p Initiated with mInputStream=%p, mOwnedStream=%p, mPlaybackStream=%p",
this, mInputStream, mOwnedStream, mPlaybackStream));
}
already_AddRefed<DOMMediaStream>
DOMMediaStream::CreateSourceStreamAsInput(nsPIDOMWindow* aWindow,
MediaStreamGraph* aGraph,
MediaStreamTrackSourceGetter* aTrackSourceGetter)
{
RefPtr<DOMMediaStream> stream = new DOMMediaStream(aWindow, aTrackSourceGetter);
stream->InitSourceStream(aGraph);
return stream.forget();
}
already_AddRefed<DOMMediaStream>
DOMMediaStream::CreateTrackUnionStreamAsInput(nsPIDOMWindow* aWindow,
MediaStreamGraph* aGraph,
MediaStreamTrackSourceGetter* aTrackSourceGetter)
{
RefPtr<DOMMediaStream> stream = new DOMMediaStream(aWindow, aTrackSourceGetter);
stream->InitTrackUnionStream(aGraph);
return stream.forget();
}
already_AddRefed<DOMMediaStream>
DOMMediaStream::CreateAudioCaptureStreamAsInput(nsPIDOMWindow* aWindow,
nsIPrincipal* aPrincipal,
MediaStreamGraph* aGraph)
{
// Audio capture doesn't create tracks dynamically
MediaStreamTrackSourceGetter* getter = nullptr;
RefPtr<DOMMediaStream> stream = new DOMMediaStream(aWindow, getter);
stream->InitAudioCaptureStream(aPrincipal, aGraph);
return stream.forget();
}
void
DOMMediaStream::PrincipalChanged(MediaStreamTrack* aTrack)
{
MOZ_ASSERT(aTrack);
NS_ASSERTION(HasTrack(*aTrack), "Principal changed for an unknown track");
LOG(LogLevel::Info, ("DOMMediaStream %p Principal changed for track %p",
this, aTrack));
RecomputePrincipal();
}
void
DOMMediaStream::RecomputePrincipal()
{
nsCOMPtr<nsIPrincipal> previousPrincipal = mPrincipal.forget();
nsCOMPtr<nsIPrincipal> previousVideoPrincipal = mVideoPrincipal.forget();
if (mTracksPendingRemoval > 0) {
LOG(LogLevel::Info, ("DOMMediaStream %p RecomputePrincipal() Cannot "
"recompute stream principal with tracks pending "
"removal.", this));
return;
}
LOG(LogLevel::Debug, ("DOMMediaStream %p Recomputing principal. "
"Old principal was %p.", this, previousPrincipal.get()));
// mPrincipal is recomputed based on all current tracks, and tracks that have
// not ended in our playback stream.
for (const RefPtr<TrackPort>& info : mTracks) {
if (info->GetTrack()->Ended()) {
continue;
}
LOG(LogLevel::Debug, ("DOMMediaStream %p Taking live track %p with "
"principal %p into account.", this,
info->GetTrack(), info->GetTrack()->GetPrincipal()));
nsContentUtils::CombineResourcePrincipals(&mPrincipal,
info->GetTrack()->GetPrincipal());
if (info->GetTrack()->AsVideoStreamTrack()) {
nsContentUtils::CombineResourcePrincipals(&mVideoPrincipal,
info->GetTrack()->GetPrincipal());
}
}
LOG(LogLevel::Debug, ("DOMMediaStream %p new principal is %p.",
this, mPrincipal.get()));
if (previousPrincipal != mPrincipal ||
previousVideoPrincipal != mVideoPrincipal) {
NotifyPrincipalChanged();
}
}
void
DOMMediaStream::NotifyPrincipalChanged()
{
if (!mPrincipal) {
// When all tracks are removed, mPrincipal will change to nullptr.
LOG(LogLevel::Info, ("DOMMediaStream %p Principal changed to nothing.",
this));
} else {
LOG(LogLevel::Info, ("DOMMediaStream %p Principal changed. Now: "
"null=%d, codebase=%d, expanded=%d, system=%d", this,
mPrincipal->GetIsNullPrincipal(),
mPrincipal->GetIsCodebasePrincipal(),
mPrincipal->GetIsExpandedPrincipal(),
mPrincipal->GetIsSystemPrincipal()));
}
for (uint32_t i = 0; i < mPrincipalChangeObservers.Length(); ++i) {
mPrincipalChangeObservers[i]->PrincipalChanged(this);
}
}
bool
DOMMediaStream::AddPrincipalChangeObserver(
PrincipalChangeObserver<DOMMediaStream>* aObserver)
{
return mPrincipalChangeObservers.AppendElement(aObserver) != nullptr;
}
bool
DOMMediaStream::RemovePrincipalChangeObserver(
PrincipalChangeObserver<DOMMediaStream>* aObserver)
{
return mPrincipalChangeObservers.RemoveElement(aObserver);
}
MediaStreamTrack*
DOMMediaStream::CreateDOMTrack(TrackID aTrackID, MediaSegment::Type aType,
MediaStreamTrackSource* aSource)
{
MOZ_RELEASE_ASSERT(mInputStream);
MOZ_RELEASE_ASSERT(mOwnedStream);
MOZ_ASSERT(FindOwnedDOMTrack(GetInputStream(), aTrackID) == nullptr);
MediaStreamTrack* track;
switch (aType) {
case MediaSegment::AUDIO:
track = new AudioStreamTrack(this, aTrackID, aTrackID, aSource);
break;
case MediaSegment::VIDEO:
track = new VideoStreamTrack(this, aTrackID, aTrackID, aSource);
break;
default:
MOZ_CRASH("Unhandled track type");
}
LOG(LogLevel::Debug, ("DOMMediaStream %p Created new track %p with ID %u", this, track, aTrackID));
mOwnedTracks.AppendElement(
new TrackPort(mOwnedPort, track, TrackPort::InputPortOwnership::EXTERNAL));
mTracks.AppendElement(
new TrackPort(mPlaybackPort, track, TrackPort::InputPortOwnership::EXTERNAL));
NotifyTrackAdded(track);
return track;
}
already_AddRefed<MediaStreamTrack>
DOMMediaStream::CloneDOMTrack(MediaStreamTrack& aTrack,
TrackID aCloneTrackID)
{
MOZ_RELEASE_ASSERT(mOwnedStream);
MOZ_RELEASE_ASSERT(mPlaybackStream);
MOZ_RELEASE_ASSERT(IsTrackIDExplicit(aCloneTrackID));
TrackID inputTrackID = aTrack.mInputTrackID;
MediaStream* inputStream = aTrack.GetInputStream();
RefPtr<MediaStreamTrack> newTrack = aTrack.CloneInternal(this, aCloneTrackID);
newTrack->mOriginalTrack =
aTrack.mOriginalTrack ? aTrack.mOriginalTrack.get() : &aTrack;
LOG(LogLevel::Debug, ("DOMMediaStream %p Created new track %p cloned from stream %p track %d",
this, newTrack.get(), inputStream, inputTrackID));
RefPtr<MediaInputPort> inputPort =
mOwnedStream->AllocateInputPort(inputStream, inputTrackID, aCloneTrackID);
mOwnedTracks.AppendElement(
new TrackPort(inputPort, newTrack, TrackPort::InputPortOwnership::OWNED));
mTracks.AppendElement(
new TrackPort(mPlaybackPort, newTrack, TrackPort::InputPortOwnership::EXTERNAL));
NotifyTrackAdded(newTrack);
newTrack->SetEnabled(aTrack.Enabled());
return newTrack.forget();
}
static DOMMediaStream::TrackPort*
FindTrackPortAmongTracks(const MediaStreamTrack& aTrack,
const nsTArray<RefPtr<DOMMediaStream::TrackPort>>& aTracks)
{
for (const RefPtr<DOMMediaStream::TrackPort>& info : aTracks) {
if (info->GetTrack() == &aTrack) {
return info;
}
}
return nullptr;
}
MediaStreamTrack*
DOMMediaStream::FindOwnedDOMTrack(MediaStream* aInputStream,
TrackID aInputTrackID) const
{
MOZ_RELEASE_ASSERT(mOwnedStream);
for (const RefPtr<TrackPort>& info : mOwnedTracks) {
if (info->GetInputPort() &&
info->GetInputPort()->GetSource() == aInputStream &&
info->GetTrack()->mInputTrackID == aInputTrackID) {
// This track is owned externally but in our playback stream.
return info->GetTrack();
}
}
return nullptr;
}
DOMMediaStream::TrackPort*
DOMMediaStream::FindOwnedTrackPort(const MediaStreamTrack& aTrack) const
{
return FindTrackPortAmongTracks(aTrack, mOwnedTracks);
}
MediaStreamTrack*
DOMMediaStream::FindPlaybackDOMTrack(MediaStream* aInputStream, TrackID aInputTrackID) const
{
if (!mPlaybackStream) {
// One would think we can assert mPlaybackStream here, but track clones have
// a dummy DOMMediaStream that doesn't have a playback stream, so we can't.
return nullptr;
}
for (const RefPtr<TrackPort>& info : mTracks) {
if (info->GetInputPort() == mPlaybackPort &&
aInputStream == mOwnedStream &&
info->GetTrack()->mInputTrackID == aInputTrackID) {
// This track is in our owned and playback streams.
return info->GetTrack();
}
if (info->GetInputPort() &&
info->GetInputPort()->GetSource() == aInputStream &&
info->GetSourceTrackId() == aInputTrackID) {
// This track is owned externally but in our playback stream.
MOZ_ASSERT(IsTrackIDExplicit(aInputTrackID));
return info->GetTrack();
}
}
return nullptr;
}
DOMMediaStream::TrackPort*
DOMMediaStream::FindPlaybackTrackPort(const MediaStreamTrack& aTrack) const
{
return FindTrackPortAmongTracks(aTrack, mTracks);
}
void
DOMMediaStream::NotifyMediaStreamGraphShutdown()
{
// No more tracks will ever be added, so just clear these callbacks now
// to prevent leaks.
mNotifiedOfMediaStreamGraphShutdown = true;
mRunOnTracksAvailable.Clear();
mTrackListeners.Clear();
mConsumersToKeepAlive.Clear();
}
void
DOMMediaStream::NotifyStreamFinished()
{
MOZ_ASSERT(IsFinished());
mConsumersToKeepAlive.Clear();
}
void
DOMMediaStream::OnTracksAvailable(OnTracksAvailableCallback* aRunnable)
{
if (mNotifiedOfMediaStreamGraphShutdown) {
// No more tracks will ever be added, so just delete the callback now.
delete aRunnable;
return;
}
mRunOnTracksAvailable.AppendElement(aRunnable);
CheckTracksAvailable();
}
void
DOMMediaStream::NotifyTracksCreated()
{
mTracksCreated = true;
CheckTracksAvailable();
}
void
DOMMediaStream::CheckTracksAvailable()
{
if (!mTracksCreated) {
return;
}
nsTArray<nsAutoPtr<OnTracksAvailableCallback> > callbacks;
callbacks.SwapElements(mRunOnTracksAvailable);
for (uint32_t i = 0; i < callbacks.Length(); ++i) {
callbacks[i]->NotifyTracksAvailable(this);
}
}
void
DOMMediaStream::RegisterTrackListener(TrackListener* aListener)
{
MOZ_ASSERT(NS_IsMainThread());
if (mNotifiedOfMediaStreamGraphShutdown) {
// No more tracks will ever be added, so just do nothing.
return;
}
mTrackListeners.AppendElement(aListener);
}
void
DOMMediaStream::UnregisterTrackListener(TrackListener* aListener)
{
MOZ_ASSERT(NS_IsMainThread());
mTrackListeners.RemoveElement(aListener);
}
void
DOMMediaStream::NotifyTrackAdded(const RefPtr<MediaStreamTrack>& aTrack)
{
MOZ_ASSERT(NS_IsMainThread());
if (mTracksPendingRemoval > 0) {
// If there are tracks pending removal we may not degrade the current
// principals until those tracks have been confirmed removed from the
// playback stream. Instead combine with the new track and the (potentially)
// degraded principal will be calculated when it's safe.
nsContentUtils::CombineResourcePrincipals(&mPrincipal,
aTrack->GetPrincipal());
LOG(LogLevel::Debug, ("DOMMediaStream %p saw a track get added. Combining "
"its principal %p into our while waiting for pending "
"tracks to be removed. New principal is %p.",
this, aTrack->GetPrincipal(), mPrincipal.get()));
if (aTrack->AsVideoStreamTrack()) {
nsContentUtils::CombineResourcePrincipals(&mVideoPrincipal,
aTrack->GetPrincipal());
}
} else {
LOG(LogLevel::Debug, ("DOMMediaStream %p saw a track get added. "
"Recomputing principal.", this));
RecomputePrincipal();
}
aTrack->AddPrincipalChangeObserver(this);
for (int32_t i = mTrackListeners.Length() - 1; i >= 0; --i) {
mTrackListeners[i]->NotifyTrackAdded(aTrack);
}
}
void
DOMMediaStream::NotifyTrackRemoved(const RefPtr<MediaStreamTrack>& aTrack)
{
MOZ_ASSERT(NS_IsMainThread());
aTrack->RemovePrincipalChangeObserver(this);
for (int32_t i = mTrackListeners.Length() - 1; i >= 0; --i) {
mTrackListeners[i]->NotifyTrackRemoved(aTrack);
}
// Don't call RecomputePrincipal here as the track may still exist in the
// playback stream in the MediaStreamGraph. It will instead be called when the
// track has been confirmed removed by the graph. See BlockPlaybackTrack().
}
void
DOMMediaStream::CreateAndAddPlaybackStreamListener(MediaStream* aStream)
{
MOZ_ASSERT(GetCameraStream(), "I'm a hack. Only DOMCameraControl may use me.");
mPlaybackListener = new PlaybackStreamListener(this);
aStream->AddListener(mPlaybackListener);
}
void
DOMMediaStream::BlockPlaybackTrack(TrackPort* aTrack)
{
MOZ_ASSERT(aTrack);
++mTracksPendingRemoval;
RefPtr<Pledge<bool>> p = aTrack->BlockSourceTrackId(aTrack->GetTrack()->mTrackID);
RefPtr<DOMMediaStream> self = this;
p->Then([self] (const bool& aIgnore) { self->NotifyPlaybackTrackBlocked(); },
[] (const nsresult& aIgnore) { NS_ERROR("Could not remove track from MSG"); }
);
}
void
DOMMediaStream::NotifyPlaybackTrackBlocked()
{
MOZ_ASSERT(mTracksPendingRemoval > 0,
"A track reported finished blocking more times than we asked for");
if (--mTracksPendingRemoval == 0) {
// The MediaStreamGraph has reported a track was blocked and we are not
// waiting for any further tracks to get blocked. It is now safe to
// recompute the principal based on our main thread track set state.
LOG(LogLevel::Debug, ("DOMMediaStream %p saw all tracks pending removal "
"finish. Recomputing principal.", this));
RecomputePrincipal();
}
}
DOMLocalMediaStream::~DOMLocalMediaStream()
{
if (mInputStream) {
// Make sure Listeners of this stream know it's going away
StopImpl();
}
}
JSObject*
DOMLocalMediaStream::WrapObject(JSContext* aCx, JS::Handle<JSObject*> aGivenProto)
{
return dom::LocalMediaStreamBinding::Wrap(aCx, this, aGivenProto);
}
void
DOMLocalMediaStream::Stop()
{
nsCOMPtr<nsPIDOMWindow> pWindow = do_QueryInterface(GetParentObject());
nsIDocument* document = pWindow ? pWindow->GetExtantDoc() : nullptr;
nsContentUtils::ReportToConsole(nsIScriptError::warningFlag,
NS_LITERAL_CSTRING("Media"),
document,
nsContentUtils::eDOM_PROPERTIES,
"MediaStreamStopDeprecatedWarning");
StopImpl();
}
void
DOMLocalMediaStream::StopImpl()
{
if (mInputStream && mInputStream->AsSourceStream()) {
mInputStream->AsSourceStream()->EndAllTrackAndFinish();
}
}
already_AddRefed<DOMLocalMediaStream>
DOMLocalMediaStream::CreateSourceStreamAsInput(nsPIDOMWindow* aWindow,
MediaStreamGraph* aGraph,
MediaStreamTrackSourceGetter* aTrackSourceGetter)
{
RefPtr<DOMLocalMediaStream> stream =
new DOMLocalMediaStream(aWindow, aTrackSourceGetter);
stream->InitSourceStream(aGraph);
return stream.forget();
}
already_AddRefed<DOMLocalMediaStream>
DOMLocalMediaStream::CreateTrackUnionStreamAsInput(nsPIDOMWindow* aWindow,
MediaStreamGraph* aGraph,
MediaStreamTrackSourceGetter* aTrackSourceGetter)
{
RefPtr<DOMLocalMediaStream> stream =
new DOMLocalMediaStream(aWindow, aTrackSourceGetter);
stream->InitTrackUnionStream(aGraph);
return stream.forget();
}
DOMAudioNodeMediaStream::DOMAudioNodeMediaStream(nsPIDOMWindow* aWindow, AudioNode* aNode)
: DOMMediaStream(aWindow, nullptr),
mStreamNode(aNode)
{
}
DOMAudioNodeMediaStream::~DOMAudioNodeMediaStream()
{
}
already_AddRefed<DOMAudioNodeMediaStream>
DOMAudioNodeMediaStream::CreateTrackUnionStreamAsInput(nsPIDOMWindow* aWindow,
AudioNode* aNode,
MediaStreamGraph* aGraph)
{
RefPtr<DOMAudioNodeMediaStream> stream = new DOMAudioNodeMediaStream(aWindow, aNode);
stream->InitTrackUnionStream(aGraph);
return stream.forget();
}
DOMHwMediaStream::DOMHwMediaStream(nsPIDOMWindow* aWindow)
: DOMLocalMediaStream(aWindow, nullptr)
{
#ifdef MOZ_WIDGET_GONK
if (!mWindow) {
NS_ERROR("Expected window here.");
mPrincipalHandle = PRINCIPAL_HANDLE_NONE;
return;
}
nsIDocument* doc = mWindow->GetExtantDoc();
if (!doc) {
NS_ERROR("Expected document here.");
mPrincipalHandle = PRINCIPAL_HANDLE_NONE;
return;
}
mPrincipalHandle = MakePrincipalHandle(doc->NodePrincipal());
#endif
}
DOMHwMediaStream::~DOMHwMediaStream()
{
}
already_AddRefed<DOMHwMediaStream>
DOMHwMediaStream::CreateHwStream(nsPIDOMWindow* aWindow,
OverlayImage* aImage)
{
RefPtr<DOMHwMediaStream> stream = new DOMHwMediaStream(aWindow);
MediaStreamGraph* graph =
MediaStreamGraph::GetInstance(MediaStreamGraph::SYSTEM_THREAD_DRIVER,
AudioChannel::Normal);
stream->InitSourceStream(graph);
stream->Init(stream->GetInputStream(), aImage);
return stream.forget();
}
void
DOMHwMediaStream::Init(MediaStream* stream, OverlayImage* aImage)
{
SourceMediaStream* srcStream = stream->AsSourceStream();
#ifdef MOZ_WIDGET_GONK
if (aImage) {
mOverlayImage = aImage;
} else {
Data imageData;
imageData.mOverlayId = DEFAULT_IMAGE_ID;
imageData.mSize.width = DEFAULT_IMAGE_WIDTH;
imageData.mSize.height = DEFAULT_IMAGE_HEIGHT;
mOverlayImage = new OverlayImage();
mOverlayImage->SetData(imageData);
}
#endif
if (srcStream) {
VideoSegment segment;
#ifdef MOZ_WIDGET_GONK
const StreamTime delta = STREAM_TIME_MAX; // Because MediaStreamGraph will run out frames in non-autoplay mode,
// we must give it bigger frame length to cover this situation.
RefPtr<Image> image = static_cast<Image*>(mOverlayImage.get());
mozilla::gfx::IntSize size = image->GetSize();
segment.AppendFrame(image.forget(), delta, size, mPrincipalHandle);
#endif
srcStream->AddTrack(TRACK_VIDEO_PRIMARY, 0, new VideoSegment());
srcStream->AppendToTrack(TRACK_VIDEO_PRIMARY, &segment);
srcStream->AdvanceKnownTracksTime(STREAM_TIME_MAX);
}
}
int32_t
DOMHwMediaStream::RequestOverlayId()
{
#ifdef MOZ_WIDGET_GONK
return mOverlayImage->GetOverlayId();
#else
return -1;
#endif
}
void
DOMHwMediaStream::SetImageSize(uint32_t width, uint32_t height)
{
#ifdef MOZ_WIDGET_GONK
if (mOverlayImage->GetSidebandStream().IsValid()) {
OverlayImage::SidebandStreamData imgData;
imgData.mStream = mOverlayImage->GetSidebandStream();
imgData.mSize = IntSize(width, height);
mOverlayImage->SetData(imgData);
} else {
OverlayImage::Data imgData;
imgData.mOverlayId = mOverlayImage->GetOverlayId();
imgData.mSize = IntSize(width, height);
mOverlayImage->SetData(imgData);
}
#endif
SourceMediaStream* srcStream = GetInputStream()->AsSourceStream();
StreamTracks::Track* track = srcStream->FindTrack(TRACK_VIDEO_PRIMARY);
if (!track || !track->GetSegment()) {
return;
}
#ifdef MOZ_WIDGET_GONK
// Clear the old segment.
// Changing the existing content of segment is a Very BAD thing, and this way will
// confuse consumers of MediaStreams.
// It is only acceptable for DOMHwMediaStream
// because DOMHwMediaStream doesn't have consumers of TV streams currently.
track->GetSegment()->Clear();
// Change the image size.
const StreamTime delta = STREAM_TIME_MAX;
RefPtr<Image> image = static_cast<Image*>(mOverlayImage.get());
mozilla::gfx::IntSize size = image->GetSize();
VideoSegment segment;
segment.AppendFrame(image.forget(), delta, size, mPrincipalHandle);
srcStream->AppendToTrack(TRACK_VIDEO_PRIMARY, &segment);
#endif
}
void
DOMHwMediaStream::SetOverlayImage(OverlayImage* aImage)
{
if (!aImage) {
return;
}
#ifdef MOZ_WIDGET_GONK
mOverlayImage = aImage;
#endif
SourceMediaStream* srcStream = GetInputStream()->AsSourceStream();
StreamTracks::Track* track = srcStream->FindTrack(TRACK_VIDEO_PRIMARY);
if (!track || !track->GetSegment()) {
return;
}
#ifdef MOZ_WIDGET_GONK
// Clear the old segment.
// Changing the existing content of segment is a Very BAD thing, and this way will
// confuse consumers of MediaStreams.
// It is only acceptable for DOMHwMediaStream
// because DOMHwMediaStream doesn't have consumers of TV streams currently.
track->GetSegment()->Clear();
// Change the image size.
const StreamTime delta = STREAM_TIME_MAX;
RefPtr<Image> image = static_cast<Image*>(mOverlayImage.get());
mozilla::gfx::IntSize size = image->GetSize();
VideoSegment segment;
segment.AppendFrame(image.forget(), delta, size, mPrincipalHandle);
srcStream->AppendToTrack(TRACK_VIDEO_PRIMARY, &segment);
#endif
}
void
DOMHwMediaStream::SetOverlayId(int32_t aOverlayId)
{
#ifdef MOZ_WIDGET_GONK
OverlayImage::Data imgData;
imgData.mOverlayId = aOverlayId;
imgData.mSize = mOverlayImage->GetSize();
mOverlayImage->SetData(imgData);
#endif
}