KotJohansson/palemoon27
1
0
Fork 0
mirror of https://github.com/roytam1/palemoon27.git synced 2026-05-26 05:37:11 +00:00
Code Issues Projects Releases Wiki Activity

import changes from `dev' branch of rmottola/Arctic-Fox:

Browse Source 
- Bug 1161101 - Use JS_GetOwnPropertyDescriptor instead of GetPropertyDescriptor in dom/bindings. r=bz (918d7961c)
- Bug 1119387 part 5: Update docker images. (0d47fde65)
- Bug 1144463 - Add dolphin-512 and update docker. r=jlal,wcosta (aa6a55cab)
- Bug 1134226: Add lightsaber nightly builds. (7926eff96)
- Bug 1151981 - Properly check for MSVC (mingw fixup). (d95e17266)
- Bug 1154947 part 3: Add aries nightly user, userdebug and eng builds. (45e096ed6)
- Bug 1178932 - Implement Reflect.construct. r=Waldo. (4c5f0e72e)
- Bug 1131206 - Remove the less useful commands from taskcluster mach r=garndt (e4c8ed78a)
- Bug 1131206 - Select mozharness version from in tree r=garndt (fdab5140d)
- bug 1156816 - Fix scopes for aws-provisioner. Switch nightly builds to production balrog. r=garndt (561d5cd58)
- Bug 1166073 - change docker registry to hub r=garndt (17faea355)
- Bug 1166233: Bring taskcluster-build mach command back. (c0a719c6b)
- Bug 1142779 - Add buildbot steps for cloning and running tests r=lightsofapollo (304235375)
- Bug 1142779 - Update job tasks to include functional unit and dual sim r=lightsofapollo (09b320602)
- Bug 1147605 - Disable tests on phone builds r=me CLOSED TREE (0912de6ba)
- Bug 1142779 - Update job tasks to use new chunking logic r=lightsofapollo (70c531920)
- Bug 1144994 part 1: Update provisioner. (2447affa4)
- Bug 1144994 part 2: Switch aries nightlies from cypress to m-c. (3019a6878)
- Bug 1164939: Provide flame-kk user, userdebug and eng builds. (ab7ad22a7)
- goanna->gecko (70b49bb18)
- Bug 1166745: Reorganize tasks. (d3090e0a9)
- Bug 1170378: Create tasks timestamps in UTC. r=jonasfj a=jonasfj (60160f061)
- Bug 987902 - Add a "doctor" mach command; r=gps (be60e7df6)
- Bug 985857 - Automatically log mach output and add a command to display it. r=gps (90993b77f)
- Bug 991983 - Add a ContextDerivedTypedListWithItems type. r=gps (ad9482c7b)
- Bug 991983 - Don't pass template paths to contexts. r=gps (30a4f2038)
- Bug 991983 - Refactor SourcePath handling for moz.build, add new classes and extensive tests. r=gps (00aeb401b)
- Bug 1142494 - Only package the steeplechase tests if webrtc is enabled. r=ted, f=drno (760943034)
- Bug 1142494 - Fix OSX packaging mistake. r=glandium, a=bustage (03cd3ca35)
- Bug 1147029 - Land luciddream in-tree, r=ted (8a4d2651b)
- Bug 1147031 - Write mach command for luciddream. r=jgriffin (b913c4df4)
- Bug 1171446 - Add a description to the luciddream mach target r=ochameau (c807799b6)
- Bug 1176642 - Remove unused imports; r=glandium (b0e458f5d)
- Bug 1151080: Rewrite NR_async_timer_set(0) to use direct dispatch. r=mt (6c3dda54d)
- Bug 1152185 - Include port/generic/include also for mtransport/test and webrtc/signaling/test. r=ekr (969ce4205)
- Bug 1156621 - Don't assume --without-system-nspr. r=glandium (4cf9c3e76)
- Bug 1035468: A NAT simulator based on NrSocket, and integrate into ice_unittest. r=ekr (903d8f6c8)
- Bug 1162026 - move WrapRunnable &co over to variadic templates; r=ekr (6224de8e9)
- Bug 1163328 - Add a Tuple class to MFBT. r=froydnj (ba3276ef3)
- Bug 1163328 - Add an And<...> class to TemplateLib.h which performs logical and on a variadic number of booleans known at compile time. r=froydnj (ac3afcabd)
- Bug 1175621 - make WrapRunnable* more efficient by utilizing moves in wrapper functions; r=ekr (15cf9f55a)
This commit is contained in:
roytam1
2020-11-27 09:26:17 +08:00
parent 9df8b92171
commit b383d6bc2f
134 changed files with 5468 additions and 2872 deletions
Download Patch File Download Diff File Expand all files Collapse all files
build/mach_bootstrap.py
+2
View File
@@ -53,6 +53,7 @@ SEARCH_PATHS = [
'testing/marionette/client',
'testing/marionette/transport',
'testing/marionette/driver',
'testing/luciddream',
'testing/mozbase/mozcrash',
'testing/mozbase/mozdebug',
'testing/mozbase/mozdevice',
@@ -84,6 +85,7 @@ MACH_MODULES = [
'python/mozbuild/mozbuild/compilation/codecomplete.py',
'python/mozbuild/mozbuild/frontend/mach_commands.py',
'services/common/tests/mach_commands.py',
'testing/luciddream/mach_commands.py',
'testing/mach_commands.py',
'testing/taskcluster/mach_commands.py',
'testing/marionette/mach_commands.py',
build/valgrind/mach_commands.py
-1
View File
@@ -5,7 +5,6 @@
from __future__ import print_function, unicode_literals
import os
import re
import subprocess
from mach.decorators import (
dom/bindings/Codegen.py
+1 -1
View File
@@ -10339,7 +10339,7 @@ class CGDOMJSProxyHandler_getOwnPropDescriptor(ClassMethod):
$*{getIndexed}
JS::Rooted<JSObject*> expando(cx);
if (!isXray && (expando = GetExpandoObject(proxy))) {
if (!JS_GetPropertyDescriptorById(cx, expando, id, desc)) {
if (!JS_GetOwnPropertyDescriptorById(cx, expando, id, desc)) {
return false;
}
if (desc.object()) {
dom/bindings/DOMJSProxyHandler.cpp
+3 -16
View File
@@ -239,25 +239,12 @@ DOMProxyHandler::set(JSContext *cx, Handle<JSObject*> proxy, Handle<jsid> id,
// Make sure to ignore our named properties when checking for own
// property descriptors for a set.
JS::Rooted<JSPropertyDescriptor> desc(cx);
JS::Rooted<JSPropertyDescriptor> ownDesc(cx);
if (!getOwnPropDescriptor(cx, proxy, id, /* ignoreNamedProps = */ true,
&desc)) {
&ownDesc)) {
return false;
}
if (!desc.object()) {
// Don't just use getPropertyDescriptor, unlike BaseProxyHandler::set,
// because that would call getOwnPropertyDescriptor on ourselves. Instead,
// directly delegate to the proto, if any.
JS::Rooted<JSObject*> proto(cx);
if (!js::GetObjectProto(cx, proxy, &proto)) {
return false;
}
if (proto && !JS_GetPropertyDescriptorById(cx, proto, id, &desc)) {
return false;
}
}
return js::SetPropertyIgnoringNamedGetter(cx, proxy, id, v, receiver, desc, result);
return js::SetPropertyIgnoringNamedGetter(cx, proxy, id, v, receiver, ownDesc, result);
}
bool
js/src/builtin/Reflect.cpp
+40 -1
View File
@@ -83,6 +83,45 @@ Reflect_apply(JSContext* cx, unsigned argc, Value* vp)
return true;
}
/* ES6 26.1.2 Reflect.construct(target, argumentsList [, newTarget]) */
static bool
Reflect_construct(JSContext* cx, unsigned argc, Value* vp)
{
CallArgs args = CallArgsFromVp(argc, vp);
// Step 1.
if (!IsConstructor(args.get(0))) {
JS_ReportErrorNumber(cx, GetErrorMessage, nullptr, JSMSG_NOT_CONSTRUCTOR,
"Reflect.construct argument");
return false;
}
// Steps 2-3.
RootedValue newTarget(cx, args.get(0));
if (argc > 2) {
newTarget = args[2];
if (!IsConstructor(newTarget)) {
JS_ReportErrorNumber(cx, GetErrorMessage, nullptr, JSMSG_NOT_CONSTRUCTOR,
"Reflect.construct argument 3");
return false;
}
}
// Step 4-5.
InvokeArgs invokeArgs(cx);
if (!InitArgsFromArrayLike(cx, args.get(1), &invokeArgs, true))
return false;
invokeArgs.setCallee(args.get(0));
invokeArgs.setThis(MagicValue(JS_THIS_POISON));
invokeArgs.newTarget().set(newTarget);
// Step 6.
if (!InvokeConstructor(cx, invokeArgs))
return false;
args.rval().set(invokeArgs.rval());
return true;
}
/* ES6 26.1.3 Reflect.defineProperty(target, propertyKey, attributes) */
static bool
Reflect_defineProperty(JSContext* cx, unsigned argc, Value* vp)
@@ -348,7 +387,7 @@ Reflect_setPrototypeOf(JSContext* cx, unsigned argc, Value* vp)
static const JSFunctionSpec methods[] = {
JS_FN("apply", Reflect_apply, 3, 0),
// JS_FN("construct", Reflect_construct, 2, 0),
JS_FN("construct", Reflect_construct, 2, 0),
JS_FN("defineProperty", Reflect_defineProperty, 3, 0),
JS_FN("deleteProperty", Reflect_deleteProperty, 2, 0),
// JS_FN("enumerate", Reflect_enumerate, 1, 0),
js/src/jsgc.h
+1 -1
View File
@@ -187,7 +187,7 @@ CanBeFinalizedInBackground(AllocKind kind, const Class* clasp)
// GCC and Clang require an explicit template declaration in front of the
// specialization of operator() because it is a dependent template. MSVC, on
// the other hand, gets very confused if we have a |template| token there.
#ifdef XP_WIN
#ifdef _MSC_VER
# define DEPENDENT_TEMPLATE_HINT
#else
# define DEPENDENT_TEMPLATE_HINT template
js/src/tests/ecma_6/Reflect/argumentsList.js
+8 -13
View File
@@ -3,28 +3,23 @@
// Tests for the argumentList argument to Reflect.apply and Reflect.construct.
if (Reflect.construct) {
throw new Error("Congratulations on implementing Reflect.construct! " +
"Please uncomment the Reflect.construct tests below.");
}
// Reflect.apply and Reflect.construct require an argumentList argument that must be an object.
assertThrowsInstanceOf(() => Reflect.apply(Math.min, undefined), // missing
TypeError);
//assertThrowsInstanceOf(() => Reflect.construct(Object), // missing
// TypeError);
assertThrowsInstanceOf(() => Reflect.construct(Object), // missing
TypeError);
for (var primitive of SOME_PRIMITIVE_VALUES) {
assertThrowsInstanceOf(() => Reflect.apply(Math.min, undefined, primitive),
TypeError);
//assertThrowsInstanceOf(() => Reflect.construct(Object, primitive),
// TypeError);
assertThrowsInstanceOf(() => Reflect.construct(Object, primitive),
TypeError);
}
// Array used by several tests below.
var BOTH = [
Reflect.apply,
// Adapt Reflect.construct to accept the same arguments as Reflect.apply.
//(target, thisArgument, argumentList) => Reflect.construct(target, argumentList)
(target, thisArgument, argumentList) => Reflect.construct(target, argumentList)
];
// The argumentList is copied and becomes the list of arguments passed to the function.
@@ -50,7 +45,7 @@ function testLess(a, b, c, d, e) {
}
args = [1, true, "three", Symbol.for];
assertEq(Reflect.apply(testLess, undefined, args), "ok");
//assertEq(Reflect.construct(testLess, args) instanceof testLess, true);
assertEq(Reflect.construct(testLess, args) instanceof testLess, true);
// argumentList.length can be more than func.length.
function testMoar(a) {
@@ -58,7 +53,7 @@ function testMoar(a) {
return "good";
}
assertEq(Reflect.apply(testMoar, undefined, args), "good");
//assertEq(Reflect.construct(testMoar, args) instanceof testMoar, true);
assertEq(Reflect.construct(testMoar, args) instanceof testMoar, true);
// argumentList can be any object with a .length property.
function getArgs(...args) {
@@ -163,6 +158,6 @@ function testMany(...args) {
return this;
}
assertEq(Reflect.apply(testMany, "pass", args), "pass");
//assertEq(Reflect.construct(testMany, args) instanceof testMany, true);
assertEq(Reflect.construct(testMany, args) instanceof testMany, true);
reportCompare(0, 0);
js/src/tests/ecma_6/Reflect/construct.js
+128
View File
@@ -0,0 +1,128 @@
/* Any copyright is dedicated to the Public Domain.
* http://creativecommons.org/licenses/publicdomain/ */
// Reflect.construct invokes constructors.
assertDeepEq(Reflect.construct(Object, []), {});
assertDeepEq(Reflect.construct(String, ["hello"]), new String("hello"));
// Constructing Date creates a real Date object.
var d = Reflect.construct(Date, [1776, 6, 4]);
assertEq(d instanceof Date, true);
assertEq(d.getFullYear(), 1776); // non-generic method requires real Date object
// [[Construct]] methods don't necessarily create new objects.
var obj = {};
assertEq(Reflect.construct(Object, [obj]), obj);
// === Various kinds of constructors
// We've already seen some builtin constructors.
//
// JS functions:
function f(x) { this.x = x; }
assertDeepEq(Reflect.construct(f, [3]), new f(3));
f.prototype = Array.prototype;
assertDeepEq(Reflect.construct(f, [3]), new f(3));
// Bound functions:
var bound = f.bind(null, "carrot");
assertDeepEq(Reflect.construct(bound, []), new bound);
// Classes:
function classesEnabled(testCode = "class Foo { constructor() {} }") {
try {
new Function(testCode);
return true;
} catch (e) {
if (!(e instanceof SyntaxError))
throw e;
return false;
}
}
if (classesEnabled()) {
eval(`{
class Base {
constructor(...args) {
this.args = args;
this.newTarget = new.target;
}
}
//class Derived extends Base {
// constructor(...args) { super(...args); }
//}
assertDeepEq(Reflect.construct(Base, []), new Base);
//assertDeepEq(Reflect.construct(Derived, [7]), new Derived(7));
//g = Derived.bind(null, "q");
//assertDeepEq(Reflect.construct(g, [8, 9]), new g(8, 9));
}`);
if (classesEnabled("class X extends Y { constructor() { super(); } }")) {
throw new Error("Congratulations on implementing super()! " +
"Please uncomment the Derived tests in this file!");
}
}
// Cross-compartment wrappers:
var g = newGlobal();
var local = {here: this};
g.eval("function F(arg) { this.arg = arg }");
assertDeepEq(Reflect.construct(g.F, [local]), new g.F(local));
// If first argument to Reflect.construct isn't a constructor, it throws a
// TypeError.
var nonConstructors = [
{},
Reflect.construct, // builtin functions aren't constructors
x => x + 1,
Math.max.bind(null, 0), // bound non-constructors aren't constructors
((x, y) => x > y).bind(null, 0),
// A Proxy to a non-constructor function isn't a constructor, even if a
// construct handler is present.
new Proxy(Reflect.construct, {construct(){}}),
];
for (var obj of nonConstructors) {
assertThrowsInstanceOf(() => Reflect.construct(obj, []), TypeError);
assertThrowsInstanceOf(() => Reflect.construct(obj, [], Object), TypeError);
}
// === new.target tests
// If the newTarget argument to Reflect.construct is missing, the target is used.
function checkNewTarget() {
assertEq(new.target, expected);
expected = undefined;
}
var expected = checkNewTarget;
Reflect.construct(checkNewTarget, []);
// The newTarget argument is correctly passed to the constructor.
var constructors = [Object, Function, f, bound];
for (var ctor of constructors) {
expected = ctor;
Reflect.construct(checkNewTarget, [], ctor);
assertEq(expected, undefined);
}
// The newTarget argument must be a constructor.
for (var v of SOME_PRIMITIVE_VALUES.concat(nonConstructors)) {
assertThrowsInstanceOf(() => Reflect.construct(checkNewTarget, [], v), TypeError);
}
// The builtin Array constructor uses new.target.prototype and always
// creates a real array object.
function someConstructor() {}
var result = Reflect.construct(Array, [], someConstructor);
assertEq(Reflect.getPrototypeOf(result),
Array.prototype, // should be someConstructor.prototype, per ES6 22.1.1.1 Array()
"Congratulations on implementing Array subclassing! Fix this test for +1 karma point.");
assertEq(result.length, 0);
assertEq(Array.isArray(result), true);
// For more Reflect.construct tests, see target.js and argumentsList.js.
reportCompare(0, 0);
js/src/tests/ecma_6/Reflect/surfaces.js
+1 -1
View File
@@ -18,7 +18,7 @@ for (var name in Reflect)
// The name and length of all the standard Reflect methods.
var methods = {
apply: 3,
//construct: 2,
construct: 2,
defineProperty: 3,
deleteProperty: 2,
//enumerate: 1,
js/src/tests/ecma_6/Reflect/target.js
+1 -1
View File
@@ -7,7 +7,7 @@
// that would be OK after a suitable target argument.
var methodInfo = {
apply: [undefined, []],
//construct: [[]],
construct: [[]],
defineProperty: ["x", {}],
deleteProperty: ["x"],
//enumerate: [],
media/mtransport/build/moz.build
-1
View File
@@ -12,7 +12,6 @@ EXPORTS.mtransport += [
'../nriceresolverfake.h',
'../rlogringbuffer.h',
'../runnable_utils.h',
'../runnable_utils_generated.h',
'../sigslot.h',
'../simpletokenbucket.h',
'../stun_udp_socket_filter.h',
media/mtransport/common.build
+1
View File
@@ -16,6 +16,7 @@ mtransport_lcppsrcs = [
'rlogringbuffer.cpp',
'simpletokenbucket.cpp',
'stun_udp_socket_filter.cpp',
'test_nr_socket.cpp',
'transportflow.cpp',
'transportlayer.cpp',
'transportlayerdtls.cpp',
media/mtransport/gonk_addrs.cpp
+1 -1
View File
@@ -129,7 +129,7 @@ nr_stun_get_addrs(nr_local_addr aAddrs[], int aMaxAddrs,
nsCOMPtr<nsIThread> mainThread = do_GetMainThread();
mozilla::SyncRunnable::DispatchToThread(
mainThread.get(),
mozilla::WrapRunnableNMRet(&GetInterfaces, &interfaces, &rv),
mozilla::WrapRunnableNMRet(&rv, &GetInterfaces, &interfaces),
false);
if (NS_FAILED(rv)) {
return R_FAILED;
media/mtransport/nr_socket_prsock.cpp
+3 -3
View File
@@ -570,7 +570,7 @@ int NrSocket::sendto(const void *msg, size_t len,
if (PR_GetError() == PR_WOULD_BLOCK_ERROR)
ABORT(R_WOULDBLOCK);
r_log(LOG_GENERIC, LOG_INFO, "Error in sendto %s", to->as_string);
r_log(LOG_GENERIC, LOG_INFO, "Error in sendto: %s", to->as_string);
ABORT(R_IO_ERROR);
}
@@ -591,7 +591,7 @@ int NrSocket::recvfrom(void * buf, size_t maxlen,
if (status <= 0) {
if (PR_GetError() == PR_WOULD_BLOCK_ERROR)
ABORT(R_WOULDBLOCK);
r_log(LOG_GENERIC, LOG_INFO, "Error in recvfrom");
r_log(LOG_GENERIC, LOG_INFO, "Error in recvfrom: %d", (int)PR_GetError());
ABORT(R_IO_ERROR);
}
*len=status;
@@ -1162,7 +1162,7 @@ static nr_socket_vtbl nr_socket_local_vtbl={
nr_socket_local_close
};
int nr_socket_local_create(nr_transport_addr *addr, nr_socket **sockp) {
int nr_socket_local_create(void *obj, nr_transport_addr *addr, nr_socket **sockp) {
RefPtr<NrSocketBase> sock;
// create IPC bridge for content process
media/mtransport/nr_socket_prsock.h
+4 -1
View File
@@ -114,6 +114,9 @@ public:
static TimeStamp short_term_violation_time();
static TimeStamp long_term_violation_time();
const nr_transport_addr& my_addr() const {
return my_addr_;
}
protected:
void fire_callback(int how);
@@ -161,7 +164,7 @@ public:
virtual int write(const void *msg, size_t len, size_t *written) override;
virtual int read(void* buf, size_t maxlen, size_t *len) override;
private:
protected:
virtual ~NrSocket() {
if (fd_)
PR_Close(fd_);
media/mtransport/nr_timer.cpp
+115 -29
View File
@@ -67,20 +67,15 @@ extern "C" {
namespace mozilla {
class nrappkitTimerCallback : public nsITimerCallback
{
public:
// We're going to release ourself in the callback, so we need to be threadsafe
NS_DECL_THREADSAFE_ISUPPORTS
NS_DECL_NSITIMERCALLBACK
nrappkitTimerCallback(NR_async_cb cb, void *cb_arg,
const char *function, int line)
class nrappkitCallback {
public:
nrappkitCallback(NR_async_cb cb, void *cb_arg,
const char *function, int line)
: cb_(cb), cb_arg_(cb_arg), function_(function), line_(line) {
}
virtual ~nrappkitCallback() {}
private:
virtual ~nrappkitTimerCallback() {}
virtual void Cancel() = 0;
protected:
/* additional members */
@@ -90,38 +85,113 @@ protected:
int line_;
};
class nrappkitTimerCallback : public nrappkitCallback,
public nsITimerCallback {
public:
// We're going to release ourself in the callback, so we need to be threadsafe
NS_DECL_THREADSAFE_ISUPPORTS
NS_DECL_NSITIMERCALLBACK
nrappkitTimerCallback(NR_async_cb cb, void *cb_arg,
const char *function, int line,
nsITimer *timer)
: nrappkitCallback(cb, cb_arg, function, line),
timer_(timer) {}
virtual void Cancel() override {
AddRef(); // Cancelling the timer causes the callback it holds to
// be released. AddRef() keeps us alive.
timer_->Cancel();
timer_->Release();
Release(); // Will cause deletion of this object.
}
private:
nsITimer* timer_;
virtual ~nrappkitTimerCallback() {}
};
NS_IMPL_ISUPPORTS(nrappkitTimerCallback, nsITimerCallback)
NS_IMETHODIMP nrappkitTimerCallback::Notify(nsITimer *timer) {
r_log(LOG_GENERIC, LOG_DEBUG, "Timer callback fired (set in %s:%d)",
function_.c_str(), line_);
MOZ_ASSERT(timer == timer_);
cb_(0, 0, cb_arg_);
// Allow the timer to go away.
timer->Release();
return NS_OK;
}
class nrappkitScheduledCallback : public nrappkitCallback {
public:
nrappkitScheduledCallback(NR_async_cb cb, void *cb_arg,
const char *function, int line)
: nrappkitCallback(cb, cb_arg, function, line) {}
void Run() {
if (cb_) {
cb_(0, 0, cb_arg_);
}
}
virtual void Cancel() override {
cb_ = nullptr;
}
~nrappkitScheduledCallback() {}
};
} // close namespace
using namespace mozilla;
// These timers must only be used from the STS thread.
// This function is a helper that enforces that.
static void CheckSTSThread() {
static nsCOMPtr<nsIEventTarget> GetSTSThread() {
nsresult rv;
nsCOMPtr<nsIEventTarget> sts_thread;
sts_thread = do_GetService(NS_SOCKETTRANSPORTSERVICE_CONTRACTID, &rv);
MOZ_ASSERT(NS_SUCCEEDED(rv));
return sts_thread;
}
// These timers must only be used from the STS thread.
// This function is a helper that enforces that.
static void CheckSTSThread() {
nsCOMPtr<nsIEventTarget> sts_thread = GetSTSThread();
ASSERT_ON_THREAD(sts_thread);
}
int NR_async_timer_set(int timeout, NR_async_cb cb, void *arg, char *func,
int l, void **handle) {
static int nr_async_timer_set_zero(NR_async_cb cb, void *arg,
char *func, int l,
nrappkitCallback **handle) {
nrappkitScheduledCallback* callback(new nrappkitScheduledCallback(
cb, arg, func, l));
nsresult rv = GetSTSThread()->Dispatch(WrapRunnable(
nsAutoPtr<nrappkitScheduledCallback>(callback),
&nrappkitScheduledCallback::Run),
NS_DISPATCH_NORMAL);
if (NS_FAILED(rv))
return R_FAILED;
*handle = callback;
// On exit to this function, the only strong reference to callback is in
// the Runnable. Because we are redispatching to the same thread,
// this is always safe.
return 0;
}
static int nr_async_timer_set_nonzero(int timeout, NR_async_cb cb, void *arg,
char *func, int l,
nrappkitCallback **handle) {
nsresult rv;
CheckSTSThread();
@@ -130,8 +200,9 @@ int NR_async_timer_set(int timeout, NR_async_cb cb, void *arg, char *func,
return(R_FAILED);
}
rv = timer->InitWithCallback(new nrappkitTimerCallback(cb, arg, func, l),
timeout, nsITimer::TYPE_ONE_SHOT);
nrappkitTimerCallback* callback =
new nrappkitTimerCallback(cb, arg, func, l, timer);
rv = timer->InitWithCallback(callback, timeout, nsITimer::TYPE_ONE_SHOT);
if (NS_FAILED(rv)) {
return R_FAILED;
}
@@ -139,11 +210,29 @@ int NR_async_timer_set(int timeout, NR_async_cb cb, void *arg, char *func,
// We need an AddRef here to keep the timer alive, per the spec.
timer->AddRef();
*handle = callback;
return 0;
}
int NR_async_timer_set(int timeout, NR_async_cb cb, void *arg,
char *func, int l, void **handle) {
CheckSTSThread();
nrappkitCallback *callback;
int r;
if (!timeout) {
r = nr_async_timer_set_zero(cb, arg, func, l, &callback);
} else {
r = nr_async_timer_set_nonzero(timeout, cb, arg, func, l, &callback);
}
if (r)
return r;
if (handle)
*handle = timer.get();
// Bug 818806: if we have no handle to the timer, we have no way to avoid
// it leaking (though not the callback object) if it never fires (or if
// we exit before it fires).
*handle = callback;
return 0;
}
@@ -163,11 +252,8 @@ int NR_async_timer_cancel(void *handle) {
CheckSTSThread();
nsITimer *timer = static_cast<nsITimer *>(handle);
timer->Cancel();
// Allow the timer to go away.
timer->Release();
nrappkitCallback* callback = static_cast<nrappkitCallback *>(handle);
callback->Cancel();
return 0;
}
media/mtransport/runnable_utils.h
+154 -16
View File
@@ -10,7 +10,10 @@
#define runnable_utils_h__
#include "nsThreadUtils.h"
#include "mozilla/IndexSequence.h"
#include "mozilla/Move.h"
#include "mozilla/RefPtr.h"
#include "mozilla/Tuple.h"
// Abstract base class for all of our templates
namespace mozilla {
@@ -53,24 +56,159 @@ class runnable_args_base : public nsRunnable {
NS_IMETHOD Run() = 0;
};
template<typename R>
struct RunnableFunctionCallHelper
{
template<typename FunType, typename... Args, size_t... Indices>
static R apply(FunType func, Tuple<Args...>& args, IndexSequence<Indices...>)
{
return func(Get<Indices>(args)...);
}
};
// A void specialization is needed in the case where the template instantiator
// knows we don't want to return a value, but we don't know whether the called
// function returns void or something else.
template<>
struct RunnableFunctionCallHelper<void>
{
template<typename FunType, typename... Args, size_t... Indices>
static void apply(FunType func, Tuple<Args...>& args, IndexSequence<Indices...>)
{
func(Get<Indices>(args)...);
}
};
template<typename R>
struct RunnableMethodCallHelper
{
template<typename Class, typename M, typename... Args, size_t... Indices>
static R apply(Class obj, M method, Tuple<Args...>& args, IndexSequence<Indices...>)
{
return ((*obj).*method)(Get<Indices>(args)...);
}
};
// A void specialization is needed in the case where the template instantiator
// knows we don't want to return a value, but we don't know whether the called
// method returns void or something else.
template<>
struct RunnableMethodCallHelper<void>
{
template<typename Class, typename M, typename... Args, size_t... Indices>
static void apply(Class obj, M method, Tuple<Args...>& args, IndexSequence<Indices...>)
{
((*obj).*method)(Get<Indices>(args)...);
}
};
}
// The generated file contains four major function templates
// (in variants for arbitrary numbers of arguments up to 10,
// which is why it is machine generated). The four templates
// are:
//
// WrapRunnable(o, m, ...) -- wraps a member function m of an object ptr o
// WrapRunnableRet(o, m, ..., r) -- wraps a member function m of an object ptr o
// the function returns something that can
// be assigned to *r
// WrapRunnableNM(f, ...) -- wraps a function f
// WrapRunnableNMRet(f, ..., r) -- wraps a function f that returns something
// that can be assigned to *r
//
// All of these template functions return a Runnable* which can be passed
// to Dispatch().
#include "runnable_utils_generated.h"
template<typename FunType, typename... Args>
class runnable_args_func : public detail::runnable_args_base<detail::NoResult>
{
public:
// |explicit| to pacify static analysis when there are no |args|.
explicit runnable_args_func(FunType f, Args&&... args)
: mFunc(f), mArgs(Forward<Args>(args)...)
{}
NS_IMETHOD Run() {
detail::RunnableFunctionCallHelper<void>::apply(mFunc, mArgs, typename IndexSequenceFor<Args...>::Type());
return NS_OK;
}
private:
FunType mFunc;
Tuple<Args...> mArgs;
};
template<typename FunType, typename... Args>
runnable_args_func<FunType, Args...>*
WrapRunnableNM(FunType f, Args... args)
{
return new runnable_args_func<FunType, Args...>(f, Move(args)...);
}
template<typename Ret, typename FunType, typename... Args>
class runnable_args_func_ret : public detail::runnable_args_base<detail::ReturnsResult>
{
public:
runnable_args_func_ret(Ret* ret, FunType f, Args&&... args)
: mReturn(ret), mFunc(f), mArgs(Forward<Args>(args)...)
{}
NS_IMETHOD Run() {
*mReturn = detail::RunnableFunctionCallHelper<Ret>::apply(mFunc, mArgs, typename IndexSequenceFor<Args...>::Type());
return NS_OK;
}
private:
Ret* mReturn;
FunType mFunc;
Tuple<Args...> mArgs;
};
template<typename R, typename FunType, typename... Args>
runnable_args_func_ret<R, FunType, Args...>*
WrapRunnableNMRet(R* ret, FunType f, Args... args)
{
return new runnable_args_func_ret<R, FunType, Args...>(ret, f, Move(args)...);
}
template<typename Class, typename M, typename... Args>
class runnable_args_memfn : public detail::runnable_args_base<detail::NoResult>
{
public:
runnable_args_memfn(Class obj, M method, Args&&... args)
: mObj(obj), mMethod(method), mArgs(Forward<Args>(args)...)
{}
NS_IMETHOD Run() {
detail::RunnableMethodCallHelper<void>::apply(mObj, mMethod, mArgs, typename IndexSequenceFor<Args...>::Type());
return NS_OK;
}
private:
Class mObj;
M mMethod;
Tuple<Args...> mArgs;
};
template<typename Class, typename M, typename... Args>
runnable_args_memfn<Class, M, Args...>*
WrapRunnable(Class obj, M method, Args... args)
{
return new runnable_args_memfn<Class, M, Args...>(obj, method, Move(args)...);
}
template<typename Ret, typename Class, typename M, typename... Args>
class runnable_args_memfn_ret : public detail::runnable_args_base<detail::ReturnsResult>
{
public:
runnable_args_memfn_ret(Ret* ret, Class obj, M method, Args... args)
: mReturn(ret), mObj(obj), mMethod(method), mArgs(Forward<Args>(args)...)
{}
NS_IMETHOD Run() {
*mReturn = detail::RunnableMethodCallHelper<Ret>::apply(mObj, mMethod, mArgs, typename IndexSequenceFor<Args...>::Type());
return NS_OK;
}
private:
Ret* mReturn;
Class mObj;
M mMethod;
Tuple<Args...> mArgs;
};
template<typename R, typename Class, typename M, typename... Args>
runnable_args_memfn_ret<R, Class, M, Args...>*
WrapRunnableRet(R* ret, Class obj, M method, Args... args)
{
return new runnable_args_memfn_ret<R, Class, M, Args...>(ret, obj, method, Move(args)...);
}
static inline nsresult RUN_ON_THREAD(nsIEventTarget *thread, detail::runnable_args_base<detail::NoResult> *runnable, uint32_t flags) {
return detail::RunOnThreadInternal(thread, static_cast<nsIRunnable *>(runnable), flags);
media/mtransport/runnable_utils.py
-195
View File
@@ -1,195 +0,0 @@
# This Source Code Form is subject to the terms of the Mozilla Public
# License, v. 2.0. If a copy of the MPL was not distributed with this
# file, You can obtain one at http://mozilla.org/MPL/2.0/.
MAX_ARGS = 15
boilerplate = "/* This Source Code Form is subject to the terms of the Mozilla Public\n\
* License, v. 2.0. If a copy of the MPL was not distributed with this\n\
* file, You can obtain one at http://mozilla.org/MPL/2.0/. */\n"
def get_args_count(args, member):
if member:
return args + 2
return args + 1
def gen_args_type(args, member):
if member:
ret = ["C o"]
else:
ret = []
ret.append("M m")
for arg in range(0, args):
ret.append("A%d a%d"%(arg, arg))
return ", ".join(ret)
def gen_args(args, member):
if member:
ret = ["o"]
else:
ret = []
ret.append("m")
for arg in range(0, args):
ret.append("a%d"%(arg))
return ", ".join(ret)
def gen_args_(args):
ret = []
for arg in range(0, args):
ret.append("a%d_"%(arg))
return ", ".join(ret)
def gen_init(args, r = False, member = False):
if member:
ret = ["o_(o)"]
else:
ret = []
ret.append("m_(m)")
if r:
ret.append("r_(r)")
for arg in range(0, args):
ret.append("a%d_(a%d)"%(arg, arg))
return ", ".join(ret)
def gen_typenames(args, member):
if member:
ret = ["typename C"]
else:
ret = []
ret.append("typename M")
for arg in range(0, args):
ret.append("typename A%d"%(arg))
return ", ".join(ret)
def gen_types(args, member):
if member:
ret = ["C"]
else:
ret = []
ret.append("M")
for arg in range(0, args):
ret.append("A%d"%(arg))
return ", ".join(ret)
def runnable_class_name(args, ret=False, member=True):
if member:
nm = "m"
else:
nm = "nm"
if ret:
class_suffix = "_ret"
enum_specializer = "detail::ReturnsResult"
else:
class_suffix = ""
enum_specializer = "detail::NoResult"
return "runnable_args_%s_%d%s" % (nm, args, class_suffix), enum_specializer
def generate_class_template(args, ret = False, member = True):
print "// %d arguments --"%args
class_name, specializer = runnable_class_name(args, ret, member)
base_class = "detail::runnable_args_base<%s>" % specializer
if not ret:
print "template<"+ gen_typenames(args, member) + "> class %s : public %s {" % (class_name, base_class)
else:
print "template<"+ gen_typenames(args, member) + ", typename R> class %s : public %s {" % (class_name, base_class)
print " public:"
if not ret:
explicit = ""
if get_args_count(args, member) == 1:
explicit = "explicit "
print " %s%s(" % (explicit, class_name) + gen_args_type(args, member) + ") :"
print " " + gen_init(args, False, member) + " {}"
else:
print " %s(" % class_name + gen_args_type(args, member) + ", R *r) :"
print " " + gen_init(args, True, member) + " {}"
print
print " NS_IMETHOD Run() {"
if ret:
print " *r_ =",
else:
print " ",
if member:
print "((*o_).*m_)(" + gen_args_(args) + ");"
else:
print "m_(" + gen_args_(args) + ");"
print " return NS_OK;"
print " }"
print
print " private:"
if member:
print " C o_;"
print " M m_;"
if ret:
print " R* r_;"
for arg in range(0, args):
print " A%d a%d_;"%(arg, arg)
print "};"
print
print
print
def generate_function_template(args, member):
if member:
NM = "";
else:
NM = "NM";
class_name, _ = runnable_class_name(args, False, member)
print "// %d arguments --"%args
print "template<" + gen_typenames(args, member) + ">"
print "%s<" % class_name + gen_types(args, member) + ">* WrapRunnable%s("%NM + gen_args_type(args, member) + ") {"
print " return new %s<" % class_name + gen_types(args, member) + ">"
print " (" + gen_args(args, member) + ");"
print "}"
print
def generate_function_template_ret(args, member):
if member:
NM = "";
else:
NM = "NM";
class_name, _ = runnable_class_name(args, True, member)
print "// %d arguments --"%args
print "template<" + gen_typenames(args, member) + ", typename R>"
print "%s<" % class_name + gen_types(args, member) + ", R>* WrapRunnable%sRet("%NM + gen_args_type(args, member) + ", R* r) {"
print " return new %s<" % class_name + gen_types(args, member) + ", R>"
print " (" + gen_args(args, member) + ", r);"
print "}"
print
print boilerplate
print
for num_args in range (0, MAX_ARGS):
generate_class_template(num_args, False, False)
generate_class_template(num_args, True, False)
generate_class_template(num_args, False, True)
generate_class_template(num_args, True, True)
print
print
print
for num_args in range(0, MAX_ARGS):
generate_function_template(num_args, False)
generate_function_template_ret(num_args, False)
generate_function_template(num_args, True)
generate_function_template_ret(num_args, True)
media/mtransport/runnable_utils_generated.h
-1928
View File
File diff suppressed because it is too large Load Diff
media/mtransport/test/ice_unittest.cpp
+243 -14
View File
@@ -35,6 +35,9 @@
#include "rlogringbuffer.h"
#include "runnable_utils.h"
#include "stunserver.h"
#include "nr_socket_prsock.h"
#include "test_nr_socket.h"
#include "ice_ctx.h"
// TODO(bcampen@mozilla.com): Big fat hack since the build system doesn't give
// us a clean way to add object files to a single executable.
#include "stunserver.cpp"
@@ -246,13 +249,21 @@ class IceTestPeer : public sigslot::has_slots<> {
expected_remote_type_(NrIceCandidate::ICE_HOST),
trickle_mode_(TRICKLE_NONE),
trickled_(0),
simulate_ice_lite_(false) {
simulate_ice_lite_(false),
nat_(new TestNat) {
ice_ctx_->SignalGatheringStateChange.connect(
this,
&IceTestPeer::GatheringStateChange);
ice_ctx_->SignalConnectionStateChange.connect(
this,
&IceTestPeer::ConnectionStateChange);
nr_socket_factory *fac;
int r = nat_->create_socket_factory(&fac);
MOZ_ASSERT(!r);
if (!r) {
nr_ice_ctx_set_socket_factory(ice_ctx_->ctx(), fac);
}
}
~IceTestPeer() {
@@ -313,6 +324,11 @@ class IceTestPeer : public sigslot::has_slots<> {
ASSERT_TRUE(NS_SUCCEEDED(ice_ctx_->SetStunServers(stun_servers)));
}
void UseTestStunServer() {
SetStunServer(TestStunServer::GetInstance()->addr(),
TestStunServer::GetInstance()->port());
}
void SetTurnServer(const std::string addr, uint16_t port,
const std::string username,
const std::string password,
@@ -361,12 +377,31 @@ class IceTestPeer : public sigslot::has_slots<> {
nsresult res;
test_utils->sts_target()->Dispatch(
WrapRunnableRet(ice_ctx_, &NrIceCtx::StartGathering, &res),
WrapRunnableRet(&res, ice_ctx_, &NrIceCtx::StartGathering),
NS_DISPATCH_SYNC);
ASSERT_TRUE(NS_SUCCEEDED(res));
}
void UseNat() {
nat_->enabled_ = true;
}
void SetFilteringType(TestNat::NatBehavior type) {
MOZ_ASSERT(!nat_->has_port_mappings());
nat_->filtering_type_ = type;
}
void SetMappingType(TestNat::NatBehavior type) {
MOZ_ASSERT(!nat_->has_port_mappings());
nat_->mapping_type_ = type;
}
void SetBlockUdp(bool block) {
MOZ_ASSERT(!nat_->has_port_mappings());
nat_->block_udp_ = block;
}
// Get various pieces of state
std::vector<std::string> GetGlobalAttributes() {
std::vector<std::string> attrs(ice_ctx_->GetGlobalAttributes());
@@ -381,7 +416,7 @@ class IceTestPeer : public sigslot::has_slots<> {
RUN_ON_THREAD(
test_utils->sts_target(),
WrapRunnableRet(this, &IceTestPeer::GetCandidates_s, stream, &v));
WrapRunnableRet(&v, this, &IceTestPeer::GetCandidates_s, stream));
return v;
}
@@ -442,7 +477,7 @@ class IceTestPeer : public sigslot::has_slots<> {
{
bool result;
test_utils->sts_target()->Dispatch(
WrapRunnableRet(this, &IceTestPeer::is_ready_s, stream, &result),
WrapRunnableRet(&result, this, &IceTestPeer::is_ready_s, stream),
NS_DISPATCH_SYNC);
return result;
}
@@ -671,7 +706,7 @@ class IceTestPeer : public sigslot::has_slots<> {
// Now start checks
test_utils->sts_target()->Dispatch(
WrapRunnableRet(ice_ctx_, &NrIceCtx::StartChecks, &res),
WrapRunnableRet(&res, ice_ctx_, &NrIceCtx::StartChecks),
NS_DISPATCH_SYNC);
ASSERT_TRUE(NS_SUCCEEDED(res));
}
@@ -748,11 +783,10 @@ class IceTestPeer : public sigslot::has_slots<> {
std::vector<NrIceCandidatePair>* pairs) {
nsresult v;
test_utils->sts_target()->Dispatch(
WrapRunnableRet(this,
WrapRunnableRet(&v, this,
&IceTestPeer::GetCandidatePairs_s,
stream_index,
pairs,
&v),
pairs),
NS_DISPATCH_SYNC);
return v;
}
@@ -958,10 +992,9 @@ class IceTestPeer : public sigslot::has_slots<> {
void SetControlling(NrIceCtx::Controlling controlling) {
nsresult res;
test_utils->sts_target()->Dispatch(
WrapRunnableRet(ice_ctx_,
WrapRunnableRet(&res, ice_ctx_,
&NrIceCtx::SetControlling,
controlling,
&res),
controlling),
NS_DISPATCH_SYNC);
ASSERT_TRUE(NS_SUCCEEDED(res));
}
@@ -1009,6 +1042,7 @@ class IceTestPeer : public sigslot::has_slots<> {
TrickleMode trickle_mode_;
int trickled_;
bool simulate_ice_lite_;
nsRefPtr<mozilla::TestNat> nat_;
};
void SchedulableTrickleCandidate::Trickle() {
@@ -1066,6 +1100,12 @@ class IceGatherTest : public ::testing::Test {
TestStunServer::GetInstance()->port());
}
void UseTestStunServer() {
TestStunServer::GetInstance()->Reset();
peer_->SetStunServer(TestStunServer::GetInstance()->addr(),
TestStunServer::GetInstance()->port());
}
// NB: Only does substring matching, watch out for stuff like "1.2.3.4"
// matching "21.2.3.47". " 1.2.3.4 " should not have false positives.
bool StreamHasMatchingCandidate(unsigned int stream,
@@ -1085,7 +1125,12 @@ class IceGatherTest : public ::testing::Test {
class IceConnectTest : public ::testing::Test {
public:
IceConnectTest() : initted_(false) {}
IceConnectTest() :
initted_(false),
use_nat_(false),
filtering_type_(TestNat::ENDPOINT_INDEPENDENT),
mapping_type_(TestNat::ENDPOINT_INDEPENDENT),
block_udp_(false) {}
void SetUp() {
nsresult rv;
@@ -1127,8 +1172,25 @@ class IceConnectTest : public ::testing::Test {
bool Gather(unsigned int waitTime = kDefaultTimeout) {
Init(false, false);
p1_->SetStunServer(g_stun_server_address, kDefaultStunServerPort);
p2_->SetStunServer(g_stun_server_address, kDefaultStunServerPort);
if (use_nat_) {
// If we enable nat simulation, but still use a real STUN server somewhere
// on the internet, we will see failures if there is a real NAT in
// addition to our simulated one, particularly if it disallows
// hairpinning.
UseTestStunServer();
p1_->UseNat();
p2_->UseNat();
p1_->SetFilteringType(filtering_type_);
p2_->SetFilteringType(filtering_type_);
p1_->SetMappingType(mapping_type_);
p2_->SetMappingType(mapping_type_);
p1_->SetBlockUdp(block_udp_);
p2_->SetBlockUdp(block_udp_);
} else {
p1_->SetStunServer(g_stun_server_address, kDefaultStunServerPort);
p2_->SetStunServer(g_stun_server_address, kDefaultStunServerPort);
}
p1_->Gather();
p2_->Gather();
@@ -1143,6 +1205,28 @@ class IceConnectTest : public ::testing::Test {
return true;
}
void UseNat() {
use_nat_ = true;
}
void SetFilteringType(TestNat::NatBehavior type) {
filtering_type_ = type;
}
void SetMappingType(TestNat::NatBehavior type) {
mapping_type_ = type;
}
void BlockUdp() {
block_udp_ = true;
}
void UseTestStunServer() {
TestStunServer::GetInstance()->Reset();
p1_->UseTestStunServer();
p2_->UseTestStunServer();
}
void SetTurnServer(const std::string addr, uint16_t port,
const std::string username,
const std::string password,
@@ -1278,6 +1362,10 @@ class IceConnectTest : public ::testing::Test {
nsCOMPtr<nsIEventTarget> target_;
mozilla::ScopedDeletePtr<IceTestPeer> p1_;
mozilla::ScopedDeletePtr<IceTestPeer> p2_;
bool use_nat_;
TestNat::NatBehavior filtering_type_;
TestNat::NatBehavior mapping_type_;
bool block_udp_;
};
class PrioritizerTest : public ::testing::Test {
@@ -1644,6 +1732,147 @@ TEST_F(IceConnectTest, TestTrickleIceLiteOfferer) {
AssertCheckingReached();
}
TEST_F(IceConnectTest, TestGatherFullCone) {
AddStream("first", 1);
UseNat();
SetFilteringType(TestNat::ENDPOINT_INDEPENDENT);
SetMappingType(TestNat::ENDPOINT_INDEPENDENT);
ASSERT_TRUE(Gather());
}
TEST_F(IceConnectTest, TestGatherFullConeAutoPrioritize) {
Init(false, true);
AddStream("first", 1);
UseNat();
SetFilteringType(TestNat::ENDPOINT_INDEPENDENT);
SetMappingType(TestNat::ENDPOINT_INDEPENDENT);
ASSERT_TRUE(Gather());
}
TEST_F(IceConnectTest, TestConnectFullCone) {
AddStream("first", 1);
UseNat();
SetFilteringType(TestNat::ENDPOINT_INDEPENDENT);
SetMappingType(TestNat::ENDPOINT_INDEPENDENT);
SetExpectedTypes(NrIceCandidate::Type::ICE_SERVER_REFLEXIVE,
NrIceCandidate::Type::ICE_SERVER_REFLEXIVE);
ASSERT_TRUE(Gather());
Connect();
}
TEST_F(IceConnectTest, TestGatherAddressRestrictedCone) {
AddStream("first", 1);
UseNat();
SetFilteringType(TestNat::ADDRESS_DEPENDENT);
SetMappingType(TestNat::ENDPOINT_INDEPENDENT);
ASSERT_TRUE(Gather());
}
TEST_F(IceConnectTest, TestConnectAddressRestrictedCone) {
AddStream("first", 1);
UseNat();
SetFilteringType(TestNat::ADDRESS_DEPENDENT);
SetMappingType(TestNat::ENDPOINT_INDEPENDENT);
SetExpectedTypes(NrIceCandidate::Type::ICE_SERVER_REFLEXIVE,
NrIceCandidate::Type::ICE_SERVER_REFLEXIVE);
ASSERT_TRUE(Gather());
Connect();
}
TEST_F(IceConnectTest, TestGatherPortRestrictedCone) {
AddStream("first", 1);
UseNat();
SetFilteringType(TestNat::PORT_DEPENDENT);
SetMappingType(TestNat::ENDPOINT_INDEPENDENT);
ASSERT_TRUE(Gather());
}
TEST_F(IceConnectTest, TestConnectPortRestrictedCone) {
AddStream("first", 1);
UseNat();
SetFilteringType(TestNat::PORT_DEPENDENT);
SetMappingType(TestNat::ENDPOINT_INDEPENDENT);
SetExpectedTypes(NrIceCandidate::Type::ICE_SERVER_REFLEXIVE,
NrIceCandidate::Type::ICE_SERVER_REFLEXIVE);
ASSERT_TRUE(Gather());
Connect();
}
TEST_F(IceConnectTest, TestGatherSymmetricNat) {
AddStream("first", 1);
UseNat();
SetFilteringType(TestNat::PORT_DEPENDENT);
SetMappingType(TestNat::PORT_DEPENDENT);
ASSERT_TRUE(Gather());
}
TEST_F(IceConnectTest, TestConnectSymmetricNat) {
if (g_turn_server.empty())
return;
AddStream("first", 1);
UseNat();
SetFilteringType(TestNat::PORT_DEPENDENT);
SetMappingType(TestNat::PORT_DEPENDENT);
p1_->SetExpectedTypes(NrIceCandidate::Type::ICE_RELAYED,
NrIceCandidate::Type::ICE_RELAYED);
p2_->SetExpectedTypes(NrIceCandidate::Type::ICE_RELAYED,
NrIceCandidate::Type::ICE_RELAYED);
SetTurnServer(g_turn_server, kDefaultStunServerPort,
g_turn_user, g_turn_password);
ASSERT_TRUE(Gather());
Connect();
}
TEST_F(IceConnectTest, TestGatherNatBlocksUDP) {
if (g_turn_server.empty())
return;
AddStream("first", 1);
UseNat();
BlockUdp();
std::vector<NrIceTurnServer> turn_servers;
std::vector<unsigned char> password_vec(g_turn_password.begin(),
g_turn_password.end());
turn_servers.push_back(
*NrIceTurnServer::Create(g_turn_server, kDefaultStunServerPort,
g_turn_user, password_vec, kNrIceTransportTcp));
turn_servers.push_back(
*NrIceTurnServer::Create(g_turn_server, kDefaultStunServerPort,
g_turn_user, password_vec, kNrIceTransportUdp));
SetTurnServers(turn_servers);
// We have to wait for the UDP-based stuff to time out.
ASSERT_TRUE(Gather(kDefaultTimeout * 3));
}
TEST_F(IceConnectTest, TestConnectNatBlocksUDP) {
if (g_turn_server.empty())
return;
AddStream("first", 1);
UseNat();
BlockUdp();
std::vector<NrIceTurnServer> turn_servers;
std::vector<unsigned char> password_vec(g_turn_password.begin(),
g_turn_password.end());
turn_servers.push_back(
*NrIceTurnServer::Create(g_turn_server, kDefaultStunServerPort,
g_turn_user, password_vec, kNrIceTransportTcp));
turn_servers.push_back(
*NrIceTurnServer::Create(g_turn_server, kDefaultStunServerPort,
g_turn_user, password_vec, kNrIceTransportUdp));
SetTurnServers(turn_servers);
p1_->SetExpectedTypes(NrIceCandidate::Type::ICE_RELAYED,
NrIceCandidate::Type::ICE_RELAYED,
kNrIceTransportTcp);
p2_->SetExpectedTypes(NrIceCandidate::Type::ICE_RELAYED,
NrIceCandidate::Type::ICE_RELAYED,
kNrIceTransportTcp);
ASSERT_TRUE(Gather(kDefaultTimeout * 3));
Connect();
}
TEST_F(IceConnectTest, TestConnectTwoComponents) {
AddStream("first", 2);
ASSERT_TRUE(Gather());
media/mtransport/test/moz.build
+15 -2
View File
@@ -14,6 +14,7 @@ if CONFIG['OS_TARGET'] != 'WINNT' and CONFIG['MOZ_WIDGET_TOOLKIT'] != 'gonk':
'runnable_utils_unittest',
'simpletokenbucket_unittest',
'sockettransportservice_unittest',
'test_nr_socket_unittest',
'TestSyncRunnable',
'transport_unittests',
'turn_unittest',
@@ -30,6 +31,8 @@ for var in ('HAVE_STRDUP', 'NR_SOCKET_IS_VOID_PTR', 'SCTP_DEBUG', 'INET'):
DEFINES[var] = True
if CONFIG['OS_TARGET'] == 'Android':
DEFINES['LINUX'] = True
DEFINES['ANDROID'] = True
LOCAL_INCLUDES += [
'/media/mtransport/third_party/nrappkit/src/port/android/include',
]
@@ -37,11 +40,13 @@ else:
DEFINES['INET6'] = True
if CONFIG['OS_TARGET'] == 'Linux':
DEFINES['LINUX'] = True
DEFINES['USE_INTERFACE_PRIORITIZER'] = True
LOCAL_INCLUDES += [
'/media/mtransport/third_party/nrappkit/src/port/linux/include',
]
USE_LIBS += [
'static:/nsprpub/lib/libc/src/plc4',
'nspr',
]
OS_LIBS += [
'-lrt',
@@ -53,13 +58,20 @@ if CONFIG['OS_TARGET'] == 'Darwin':
]
if CONFIG['OS_TARGET'] in ('DragonFly', 'FreeBSD', 'NetBSD', 'OpenBSD'):
if CONFIG['OS_TARGET'] == 'Darwin':
DEFINES['DARWIN'] = True
else:
DEFINES['BSD'] = True
LOCAL_INCLUDES += [
'/media/mtransport/third_party/nrappkit/src/port/darwin/include',
'/media/mtransport/third_party/nrappkit/src/port/generic/include',
]
# SCTP DEFINES
if CONFIG['OS_TARGET'] == 'WINNT':
DEFINES['WIN'] = True
# for stun.h
DEFINES['WIN32'] = True
DEFINES['NOMINMAX'] = True
DEFINES['__Userspace_os_Windows'] = 1
else:
# Works for Darwin, Linux, Android. Probably doesn't work for others.
@@ -79,6 +91,7 @@ LOCAL_INCLUDES += [
'/media/mtransport/third_party/nrappkit/src/event',
'/media/mtransport/third_party/nrappkit/src/log',
'/media/mtransport/third_party/nrappkit/src/plugin',
'/media/mtransport/third_party/nrappkit/src/port/generic/include',
'/media/mtransport/third_party/nrappkit/src/registry',
'/media/mtransport/third_party/nrappkit/src/share',
'/media/mtransport/third_party/nrappkit/src/stats',
media/mtransport/test/nrappkit_unittest.cpp
+29 -4
View File
@@ -38,7 +38,17 @@ class TimerTest : public ::testing::Test {
int ret;
test_utils->sts_target()->Dispatch(
WrapRunnableRet(this, &TimerTest::ArmTimer_w, timeout, &ret),
WrapRunnableRet(&ret, this, &TimerTest::ArmTimer_w, timeout),
NS_DISPATCH_SYNC);
return ret;
}
int ArmCancelTimer(int timeout) {
int ret;
test_utils->sts_target()->Dispatch(
WrapRunnableRet(&ret, this, &TimerTest::ArmCancelTimer_w, timeout),
NS_DISPATCH_SYNC);
return ret;
@@ -48,11 +58,20 @@ class TimerTest : public ::testing::Test {
return NR_ASYNC_TIMER_SET(timeout, cb, this, &handle_);
}
int ArmCancelTimer_w(int timeout) {
int r;
r = ArmTimer_w(timeout);
if (r)
return r;
return CancelTimer_w();
}
int CancelTimer() {
int ret;
test_utils->sts_target()->Dispatch(
WrapRunnableRet(this, &TimerTest::CancelTimer_w, &ret),
WrapRunnableRet(&ret, this, &TimerTest::CancelTimer_w),
NS_DISPATCH_SYNC);
return ret;
@@ -66,7 +85,7 @@ class TimerTest : public ::testing::Test {
int ret;
test_utils->sts_target()->Dispatch(
WrapRunnableRet(this, &TimerTest::Schedule_w, &ret),
WrapRunnableRet(&ret, this, &TimerTest::Schedule_w),
NS_DISPATCH_SYNC);
return ret;
@@ -74,7 +93,7 @@ class TimerTest : public ::testing::Test {
int Schedule_w() {
NR_ASYNC_SCHEDULE(cb, this);
return 0;
}
@@ -105,6 +124,12 @@ TEST_F(TimerTest, CancelTimer) {
ASSERT_FALSE(fired_);
}
TEST_F(TimerTest, CancelTimer0) {
ArmCancelTimer(0);
PR_Sleep(100);
ASSERT_FALSE(fired_);
}
TEST_F(TimerTest, ScheduleTest) {
Schedule();
ASSERT_TRUE_WAIT(fired_, 1000);
media/mtransport/test/runnable_utils_unittest.cpp
+2 -2
View File
@@ -138,7 +138,7 @@ class DispatchTest : public ::testing::Test {
int z;
int x = 10;
target_->Dispatch(WrapRunnableRet(&cl_, &TargetClass::return_int, x, &z),
target_->Dispatch(WrapRunnableRet(&z, &cl_, &TargetClass::return_int, x),
NS_DISPATCH_SYNC);
ASSERT_EQ(10, z);
}
@@ -195,7 +195,7 @@ TEST_F(DispatchTest, TestNonMethodRet) {
int z;
test_utils->sts_target()->Dispatch(
WrapRunnableNMRet(SetNonMethodRet, &cl_, 10, &z), NS_DISPATCH_SYNC);
WrapRunnableNMRet(&z, SetNonMethodRet, &cl_, 10), NS_DISPATCH_SYNC);
ASSERT_EQ(1, ran_);
ASSERT_EQ(10, z);
media/mtransport/test/stunserver.cpp
+1 -1
View File
@@ -221,7 +221,7 @@ int TestStunServer::TryOpenListenSocket(nr_local_addr* addr, uint16_t port) {
return R_INTERNAL;
}
if (nr_socket_local_create(&addr->addr, &listen_sock_)) {
if (nr_socket_local_create(nullptr, &addr->addr, &listen_sock_)) {
MOZ_MTLOG(ML_ERROR, "Couldn't create listen socket");
return R_ALREADY;
}
media/mtransport/test/test_nr_socket_unittest.cpp
+660
View File
@@ -0,0 +1,660 @@
/* -*- Mode: C++; tab-width: 8; indent-tabs-mode: nil; c-basic-offset: 2 -*- */
/* vim: set ts=2 et sw=2 tw=80: */
/* This Source Code Form is subject to the terms of the Mozilla Public
* License, v. 2.0. If a copy of the MPL was not distributed with this file,
* You can obtain one at http://mozilla.org/MPL/2.0/. */
// Original author: bcampen@mozilla.com
extern "C" {
#include "stun_msg.h" // for NR_STUN_MAX_MESSAGE_SIZE
#include "stun_util.h"
#include "nr_api.h"
#include "async_wait.h"
#include "nr_socket.h"
#include "nr_socket_local.h"
#include "stun_hint.h"
#include "local_addr.h"
#include "registry.h"
}
#include "test_nr_socket.h"
#include "nsCOMPtr.h"
#include "nsNetCID.h"
#include "nsServiceManagerUtils.h"
#include "nsAutoPtr.h"
#include "runnable_utils.h"
#include "mtransport_test_utils.h"
#include <vector>
#define GTEST_HAS_RTTI 0
#include "gtest/gtest.h"
#include "gtest_utils.h"
namespace mozilla {
class TestNrSocketTest : public ::testing::Test {
public:
TestNrSocketTest() :
wait_done_for_main_(false),
sts_(),
public_addrs_(),
private_addrs_(),
nats_() {
// Get the transport service as a dispatch target
nsresult rv;
sts_ = do_GetService(NS_SOCKETTRANSPORTSERVICE_CONTRACTID, &rv);
EXPECT_TRUE(NS_SUCCEEDED(rv)) << "Failed to get STS: " << (int)rv;
}
~TestNrSocketTest() {
sts_->Dispatch(WrapRunnable(this, &TestNrSocketTest::TearDown_s),
NS_DISPATCH_SYNC);
}
void TearDown_s() {
public_addrs_.clear();
private_addrs_.clear();
nats_.clear();
sts_ = nullptr;
}
nsRefPtr<TestNrSocket> CreateTestNrSocket_s(const char *ip_str,
TestNat *nat) {
// If no nat is supplied, we create a default NAT which is disabled. This
// is how we simulate a non-natted socket.
nsRefPtr<TestNrSocket> sock(new TestNrSocket(nat ? nat : new TestNat));
nr_transport_addr address;
nr_ip4_str_port_to_transport_addr(ip_str, 0, IPPROTO_UDP, &address);
int r = sock->create(&address);
if (r) {
return nullptr;
}
return sock;
}
void CreatePublicAddrs(size_t count, const char *ip_str = "127.0.0.1") {
sts_->Dispatch(
WrapRunnable(this,
&TestNrSocketTest::CreatePublicAddrs_s,
count,
ip_str),
NS_DISPATCH_SYNC);
}
void CreatePublicAddrs_s(size_t count, const char* ip_str) {
while (count--) {
auto sock = CreateTestNrSocket_s(ip_str, nullptr);
ASSERT_TRUE(sock) << "Failed to create socket";
public_addrs_.push_back(sock);
}
}
nsRefPtr<TestNat> CreatePrivateAddrs(size_t size,
const char* ip_str = "127.0.0.1") {
nsRefPtr<TestNat> result;
sts_->Dispatch(
WrapRunnableRet(&result,
this,
&TestNrSocketTest::CreatePrivateAddrs_s,
size,
ip_str),
NS_DISPATCH_SYNC);
return result;
}
nsRefPtr<TestNat> CreatePrivateAddrs_s(size_t count, const char* ip_str) {
nsRefPtr<TestNat> nat(new TestNat);
while (count--) {
auto sock = CreateTestNrSocket_s(ip_str, nat);
if (!sock) {
EXPECT_TRUE(false) << "Failed to create socket";
break;
}
private_addrs_.push_back(sock);
}
nat->enabled_ = true;
nats_.push_back(nat);
return nat;
}
bool CheckConnectivityVia(
TestNrSocket *from,
TestNrSocket *to,
const nr_transport_addr &via,
nr_transport_addr *sender_external_address = nullptr) {
MOZ_ASSERT(from);
if (!WaitForWriteable(from)) {
return false;
}
int result = 0;
sts_->Dispatch(WrapRunnableRet(&result,
this,
&TestNrSocketTest::SendData_s,
from,
via),
NS_DISPATCH_SYNC);
if (result) {
return false;
}
if (!WaitForReadable(to)) {
return false;
}
nr_transport_addr dummy_outparam;
if (!sender_external_address) {
sender_external_address = &dummy_outparam;
}
MOZ_ASSERT(to);
sts_->Dispatch(WrapRunnableRet(&result,
this,
&TestNrSocketTest::RecvData_s,
to,
sender_external_address),
NS_DISPATCH_SYNC);
return !result;
}
bool CheckConnectivity(
TestNrSocket *from,
TestNrSocket *to,
nr_transport_addr *sender_external_address = nullptr) {
nr_transport_addr destination_address;
int r = GetAddress(to, &destination_address);
if (r) {
return false;
}
return CheckConnectivityVia(from,
to,
destination_address,
sender_external_address);
}
int GetAddress(TestNrSocket *sock, nr_transport_addr_ *address) {
MOZ_ASSERT(sock);
MOZ_ASSERT(address);
int r;
sts_->Dispatch(WrapRunnableRet(&r,
this,
&TestNrSocketTest::GetAddress_s,
sock,
address),
NS_DISPATCH_SYNC);
return r;
}
int GetAddress_s(TestNrSocket *sock, nr_transport_addr *address) {
return sock->getaddr(address);
}
int SendData_s(TestNrSocket *from, const nr_transport_addr &to) {
// It is up to caller to ensure that |from| is writeable.
const char buf[] = "foobajooba";
return from->sendto(buf, sizeof(buf), 0,
// TODO(bug 1170299): Remove const_cast when no longer necessary
const_cast<nr_transport_addr*>(&to));
}
int RecvData_s(TestNrSocket *to, nr_transport_addr *from) {
// It is up to caller to ensure that |to| is readable
const size_t bufSize = 1024;
char buf[bufSize];
size_t len;
// Maybe check that data matches?
int r = to->recvfrom(buf, sizeof(buf), &len, 0, from);
if (!r && (len == 0)) {
r = R_INTERNAL;
}
return r;
}
bool WaitForSocketState(TestNrSocket *sock, int state) {
MOZ_ASSERT(sock);
sts_->Dispatch(WrapRunnable(this,
&TestNrSocketTest::WaitForSocketState_s,
sock,
state),
NS_DISPATCH_SYNC);
bool res;
WAIT_(wait_done_for_main_, 100, res);
wait_done_for_main_ = false;
if (!res) {
sts_->Dispatch(WrapRunnable(this,
&TestNrSocketTest::CancelWait_s,
sock,
state),
NS_DISPATCH_SYNC);
}
return res;
}
void WaitForSocketState_s(TestNrSocket *sock, int state) {
NR_ASYNC_WAIT(sock, state, &WaitDone, this);
}
void CancelWait_s(TestNrSocket *sock, int state) {
sock->cancel(state);
}
bool WaitForReadable(TestNrSocket *sock) {
return WaitForSocketState(sock, NR_ASYNC_WAIT_READ);
}
bool WaitForWriteable(TestNrSocket *sock) {
return WaitForSocketState(sock, NR_ASYNC_WAIT_WRITE);
}
static void WaitDone(void *sock, int how, void *test_fixture) {
TestNrSocketTest *test = static_cast<TestNrSocketTest*>(test_fixture);
test->wait_done_for_main_ = true;
}
// Simple busywait boolean for the test cases to spin on.
Atomic<bool> wait_done_for_main_;
nsCOMPtr<nsIEventTarget> sts_;
std::vector<nsRefPtr<TestNrSocket>> public_addrs_;
std::vector<nsRefPtr<TestNrSocket>> private_addrs_;
std::vector<nsRefPtr<TestNat>> nats_;
};
} // namespace mozilla
using mozilla::TestNrSocketTest;
using mozilla::TestNat;
TEST_F(TestNrSocketTest, PublicConnectivity) {
CreatePublicAddrs(2);
ASSERT_TRUE(CheckConnectivity(public_addrs_[0], public_addrs_[1]));
ASSERT_TRUE(CheckConnectivity(public_addrs_[1], public_addrs_[0]));
ASSERT_TRUE(CheckConnectivity(public_addrs_[0], public_addrs_[0]));
ASSERT_TRUE(CheckConnectivity(public_addrs_[1], public_addrs_[1]));
}
TEST_F(TestNrSocketTest, PrivateConnectivity) {
nsRefPtr<TestNat> nat(CreatePrivateAddrs(2));
nat->filtering_type_ = TestNat::ENDPOINT_INDEPENDENT;
nat->mapping_type_ = TestNat::ENDPOINT_INDEPENDENT;
ASSERT_TRUE(CheckConnectivity(private_addrs_[0], private_addrs_[1]));
ASSERT_TRUE(CheckConnectivity(private_addrs_[1], private_addrs_[0]));
ASSERT_TRUE(CheckConnectivity(private_addrs_[0], private_addrs_[0]));
ASSERT_TRUE(CheckConnectivity(private_addrs_[1], private_addrs_[1]));
}
TEST_F(TestNrSocketTest, NoConnectivityWithoutPinhole) {
nsRefPtr<TestNat> nat(CreatePrivateAddrs(1));
nat->filtering_type_ = TestNat::ENDPOINT_INDEPENDENT;
nat->mapping_type_ = TestNat::ENDPOINT_INDEPENDENT;
CreatePublicAddrs(1);
ASSERT_FALSE(CheckConnectivity(public_addrs_[0], private_addrs_[0]));
}
TEST_F(TestNrSocketTest, NoConnectivityBetweenSubnets) {
nsRefPtr<TestNat> nat1(CreatePrivateAddrs(1));
nat1->filtering_type_ = TestNat::ENDPOINT_INDEPENDENT;
nat1->mapping_type_ = TestNat::ENDPOINT_INDEPENDENT;
nsRefPtr<TestNat> nat2(CreatePrivateAddrs(1));
nat2->filtering_type_ = TestNat::ENDPOINT_INDEPENDENT;
nat2->mapping_type_ = TestNat::ENDPOINT_INDEPENDENT;
ASSERT_FALSE(CheckConnectivity(private_addrs_[0], private_addrs_[1]));
ASSERT_FALSE(CheckConnectivity(private_addrs_[1], private_addrs_[0]));
ASSERT_TRUE(CheckConnectivity(private_addrs_[0], private_addrs_[0]));
ASSERT_TRUE(CheckConnectivity(private_addrs_[1], private_addrs_[1]));
}
TEST_F(TestNrSocketTest, FullConeAcceptIngress) {
nsRefPtr<TestNat> nat(CreatePrivateAddrs(1));
nat->filtering_type_ = TestNat::ENDPOINT_INDEPENDENT;
nat->mapping_type_ = TestNat::ENDPOINT_INDEPENDENT;
CreatePublicAddrs(2);
nr_transport_addr sender_external_address;
// Open pinhole to public IP 0
ASSERT_TRUE(CheckConnectivity(private_addrs_[0],
public_addrs_[0],
&sender_external_address));
// Verify that return traffic works
ASSERT_TRUE(CheckConnectivityVia(public_addrs_[0],
private_addrs_[0],
sender_external_address));
// Verify that other public IP can use the pinhole
ASSERT_TRUE(CheckConnectivityVia(public_addrs_[1],
private_addrs_[0],
sender_external_address));
}
TEST_F(TestNrSocketTest, FullConeOnePinhole) {
nsRefPtr<TestNat> nat(CreatePrivateAddrs(1));
nat->filtering_type_ = TestNat::ENDPOINT_INDEPENDENT;
nat->mapping_type_ = TestNat::ENDPOINT_INDEPENDENT;
CreatePublicAddrs(2);
nr_transport_addr sender_external_address;
// Open pinhole to public IP 0
ASSERT_TRUE(CheckConnectivity(private_addrs_[0],
public_addrs_[0],
&sender_external_address));
// Verify that return traffic works
ASSERT_TRUE(CheckConnectivityVia(public_addrs_[0],
private_addrs_[0],
sender_external_address));
// Send traffic to other public IP, verify that it uses the same pinhole
nr_transport_addr sender_external_address2;
ASSERT_TRUE(CheckConnectivity(private_addrs_[0],
public_addrs_[1],
&sender_external_address2));
ASSERT_FALSE(nr_transport_addr_cmp(&sender_external_address,
&sender_external_address2,
NR_TRANSPORT_ADDR_CMP_MODE_ALL))
<< "addr1: " << sender_external_address.as_string << " addr2: "
<< sender_external_address2.as_string;
}
// OS 10.6 doesn't seem to allow us to open ports on 127.0.0.2, and while linux
// does allow this, it has other behavior (see below) that prevents this test
// from working.
TEST_F(TestNrSocketTest, DISABLED_AddressRestrictedCone) {
nsRefPtr<TestNat> nat(CreatePrivateAddrs(1));
nat->filtering_type_ = TestNat::ADDRESS_DEPENDENT;
nat->mapping_type_ = TestNat::ENDPOINT_INDEPENDENT;
CreatePublicAddrs(2, "127.0.0.1");
CreatePublicAddrs(1, "127.0.0.2");
nr_transport_addr sender_external_address;
// Open pinhole to public IP 0
ASSERT_TRUE(CheckConnectivity(private_addrs_[0],
public_addrs_[0],
&sender_external_address));
// Verify that return traffic works
ASSERT_TRUE(CheckConnectivityVia(public_addrs_[0],
private_addrs_[0],
sender_external_address));
// Verify that another address on the same host can use the pinhole
ASSERT_TRUE(CheckConnectivityVia(public_addrs_[1],
private_addrs_[0],
sender_external_address));
// Linux has a tendency to monkey around with source addresses, doing
// stuff like substituting 127.0.0.1 for packets sent by 127.0.0.2, and even
// going as far as substituting localhost for a packet sent from a real IP
// address when the destination is localhost. The only way to make this test
// work on linux is to have two real IP addresses.
#ifndef __linux__
// Verify that an address on a different host can't use the pinhole
ASSERT_FALSE(CheckConnectivityVia(public_addrs_[2],
private_addrs_[0],
sender_external_address));
#endif
// Send traffic to other public IP, verify that it uses the same pinhole
nr_transport_addr sender_external_address2;
ASSERT_TRUE(CheckConnectivity(private_addrs_[0],
public_addrs_[1],
&sender_external_address2));
ASSERT_FALSE(nr_transport_addr_cmp(&sender_external_address,
&sender_external_address2,
NR_TRANSPORT_ADDR_CMP_MODE_ALL))
<< "addr1: " << sender_external_address.as_string << " addr2: "
<< sender_external_address2.as_string;
// Verify that the other public IP can now use the pinhole
ASSERT_TRUE(CheckConnectivityVia(public_addrs_[1],
private_addrs_[0],
sender_external_address2));
// Send traffic to other public IP, verify that it uses the same pinhole
nr_transport_addr sender_external_address3;
ASSERT_TRUE(CheckConnectivity(private_addrs_[0],
public_addrs_[2],
&sender_external_address3));
ASSERT_FALSE(nr_transport_addr_cmp(&sender_external_address,
&sender_external_address3,
NR_TRANSPORT_ADDR_CMP_MODE_ALL))
<< "addr1: " << sender_external_address.as_string << " addr2: "
<< sender_external_address3.as_string;
// Verify that the other public IP can now use the pinhole
ASSERT_TRUE(CheckConnectivityVia(public_addrs_[2],
private_addrs_[0],
sender_external_address3));
}
TEST_F(TestNrSocketTest, RestrictedCone) {
nsRefPtr<TestNat> nat(CreatePrivateAddrs(1));
nat->filtering_type_ = TestNat::PORT_DEPENDENT;
nat->mapping_type_ = TestNat::ENDPOINT_INDEPENDENT;
CreatePublicAddrs(2);
nr_transport_addr sender_external_address;
// Open pinhole to public IP 0
ASSERT_TRUE(CheckConnectivity(private_addrs_[0],
public_addrs_[0],
&sender_external_address));
// Verify that return traffic works
ASSERT_TRUE(CheckConnectivityVia(public_addrs_[0],
private_addrs_[0],
sender_external_address));
// Verify that other public IP cannot use the pinhole
ASSERT_FALSE(CheckConnectivityVia(public_addrs_[1],
private_addrs_[0],
sender_external_address));
// Send traffic to other public IP, verify that it uses the same pinhole
nr_transport_addr sender_external_address2;
ASSERT_TRUE(CheckConnectivity(private_addrs_[0],
public_addrs_[1],
&sender_external_address2));
ASSERT_FALSE(nr_transport_addr_cmp(&sender_external_address,
&sender_external_address2,
NR_TRANSPORT_ADDR_CMP_MODE_ALL))
<< "addr1: " << sender_external_address.as_string << " addr2: "
<< sender_external_address2.as_string;
// Verify that the other public IP can now use the pinhole
ASSERT_TRUE(CheckConnectivityVia(public_addrs_[1],
private_addrs_[0],
sender_external_address2));
}
TEST_F(TestNrSocketTest, PortDependentMappingFullCone) {
nsRefPtr<TestNat> nat(CreatePrivateAddrs(1));
nat->filtering_type_ = TestNat::ENDPOINT_INDEPENDENT;
nat->mapping_type_ = TestNat::PORT_DEPENDENT;
CreatePublicAddrs(2);
nr_transport_addr sender_external_address0;
// Open pinhole to public IP 0
ASSERT_TRUE(CheckConnectivity(private_addrs_[0],
public_addrs_[0],
&sender_external_address0));
// Verify that return traffic works
ASSERT_TRUE(CheckConnectivityVia(public_addrs_[0],
private_addrs_[0],
sender_external_address0));
// Verify that other public IP can use the pinhole
ASSERT_TRUE(CheckConnectivityVia(public_addrs_[1],
private_addrs_[0],
sender_external_address0));
// Send traffic to other public IP, verify that it uses a different pinhole
nr_transport_addr sender_external_address1;
ASSERT_TRUE(CheckConnectivity(private_addrs_[0],
public_addrs_[1],
&sender_external_address1));
ASSERT_TRUE(nr_transport_addr_cmp(&sender_external_address0,
&sender_external_address1,
NR_TRANSPORT_ADDR_CMP_MODE_ALL))
<< "addr1: " << sender_external_address0.as_string << " addr2: "
<< sender_external_address1.as_string;
// Verify that return traffic works
ASSERT_TRUE(CheckConnectivityVia(public_addrs_[1],
private_addrs_[0],
sender_external_address1));
// Verify that other public IP can use the original pinhole
ASSERT_TRUE(CheckConnectivityVia(public_addrs_[0],
private_addrs_[0],
sender_external_address1));
}
TEST_F(TestNrSocketTest, Symmetric) {
nsRefPtr<TestNat> nat(CreatePrivateAddrs(1));
nat->filtering_type_ = TestNat::PORT_DEPENDENT;
nat->mapping_type_ = TestNat::PORT_DEPENDENT;
CreatePublicAddrs(2);
nr_transport_addr sender_external_address;
// Open pinhole to public IP 0
ASSERT_TRUE(CheckConnectivity(private_addrs_[0],
public_addrs_[0],
&sender_external_address));
// Verify that return traffic works
ASSERT_TRUE(CheckConnectivityVia(public_addrs_[0],
private_addrs_[0],
sender_external_address));
// Verify that other public IP cannot use the pinhole
ASSERT_FALSE(CheckConnectivityVia(public_addrs_[1],
private_addrs_[0],
sender_external_address));
// Send traffic to other public IP, verify that it uses a new pinhole
nr_transport_addr sender_external_address2;
ASSERT_TRUE(CheckConnectivity(private_addrs_[0],
public_addrs_[1],
&sender_external_address2));
ASSERT_TRUE(nr_transport_addr_cmp(&sender_external_address,
&sender_external_address2,
NR_TRANSPORT_ADDR_CMP_MODE_ALL));
// Verify that the other public IP can use the new pinhole
ASSERT_TRUE(CheckConnectivityVia(public_addrs_[1],
private_addrs_[0],
sender_external_address2));
}
TEST_F(TestNrSocketTest, BlockUdp) {
nsRefPtr<TestNat> nat(CreatePrivateAddrs(2));
nat->block_udp_ = true;
CreatePublicAddrs(1);
nr_transport_addr sender_external_address;
ASSERT_FALSE(CheckConnectivity(private_addrs_[0],
public_addrs_[0],
&sender_external_address));
// Make sure UDP behind the NAT still works
ASSERT_TRUE(CheckConnectivity(private_addrs_[0],
private_addrs_[1]));
ASSERT_TRUE(CheckConnectivity(private_addrs_[1],
private_addrs_[0]));
}
TEST_F(TestNrSocketTest, DenyHairpinning) {
nsRefPtr<TestNat> nat(CreatePrivateAddrs(2));
nat->filtering_type_ = TestNat::ENDPOINT_INDEPENDENT;
nat->mapping_type_ = TestNat::ENDPOINT_INDEPENDENT;
CreatePublicAddrs(1);
nr_transport_addr sender_external_address;
// Open pinhole to public IP 0
ASSERT_TRUE(CheckConnectivity(private_addrs_[0],
public_addrs_[0],
&sender_external_address));
// Verify that hairpinning is disallowed
ASSERT_FALSE(CheckConnectivityVia(private_addrs_[1],
private_addrs_[0],
sender_external_address));
}
TEST_F(TestNrSocketTest, AllowHairpinning) {
nsRefPtr<TestNat> nat(CreatePrivateAddrs(2));
nat->filtering_type_ = TestNat::ENDPOINT_INDEPENDENT;
nat->mapping_type_ = TestNat::ENDPOINT_INDEPENDENT;
nat->mapping_timeout_ = 30000;
nat->allow_hairpinning_ = true;
CreatePublicAddrs(1);
nr_transport_addr sender_external_address;
// Open pinhole to public IP 0, obtain external address
ASSERT_TRUE(CheckConnectivity(private_addrs_[0],
public_addrs_[0],
&sender_external_address));
// Verify that hairpinning is allowed
ASSERT_TRUE(CheckConnectivityVia(private_addrs_[1],
private_addrs_[0],
sender_external_address));
}
TEST_F(TestNrSocketTest, FullConeTimeout) {
nsRefPtr<TestNat> nat(CreatePrivateAddrs(1));
nat->filtering_type_ = TestNat::ENDPOINT_INDEPENDENT;
nat->mapping_type_ = TestNat::ENDPOINT_INDEPENDENT;
nat->mapping_timeout_ = 200;
CreatePublicAddrs(2);
nr_transport_addr sender_external_address;
// Open pinhole to public IP 0
ASSERT_TRUE(CheckConnectivity(private_addrs_[0],
public_addrs_[0],
&sender_external_address));
// Verify that return traffic works
ASSERT_TRUE(CheckConnectivityVia(public_addrs_[0],
private_addrs_[0],
sender_external_address));
PR_Sleep(201);
// Verify that return traffic does not work
ASSERT_FALSE(CheckConnectivityVia(public_addrs_[0],
private_addrs_[0],
sender_external_address));
}
// TODO(): We need TCP tests, but first we will need ICE TCP to land (this
// adds listen/accept support to NrSocket)
int main(int argc, char **argv)
{
// Inits STS and some other stuff.
MtransportTestUtils test_utils;
NR_reg_init(NR_REG_MODE_LOCAL);
// Start the tests
::testing::InitGoogleTest(&argc, argv);
int rv = RUN_ALL_TESTS();
return rv;
}
media/mtransport/test/transport_unittests.cpp
+13 -13
View File
@@ -633,7 +633,7 @@ class TransportTestPeer : public sigslot::has_slots<> {
layers->push(dtls_);
test_utils->sts_target()->Dispatch(
WrapRunnableRet(flow_, &TransportFlow::PushLayers, layers, &res),
WrapRunnableRet(&res, flow_, &TransportFlow::PushLayers, layers),
NS_DISPATCH_SYNC);
ASSERT_EQ((nsresult)NS_OK, res);
@@ -644,7 +644,7 @@ class TransportTestPeer : public sigslot::has_slots<> {
// Start gathering
test_utils->sts_target()->Dispatch(
WrapRunnableRet(ice_ctx_, &NrIceCtx::StartGathering, &res),
WrapRunnableRet(&res, ice_ctx_, &NrIceCtx::StartGathering),
NS_DISPATCH_SYNC);
ASSERT_TRUE(NS_SUCCEEDED(res));
}
@@ -684,23 +684,23 @@ class TransportTestPeer : public sigslot::has_slots<> {
// First send attributes
test_utils->sts_target()->Dispatch(
WrapRunnableRet(peer_->ice_ctx_,
WrapRunnableRet(&res, peer_->ice_ctx_,
&NrIceCtx::ParseGlobalAttributes,
ice_ctx_->GetGlobalAttributes(), &res),
ice_ctx_->GetGlobalAttributes()),
NS_DISPATCH_SYNC);
ASSERT_TRUE(NS_SUCCEEDED(res));
for (size_t i=0; i<streams_.size(); ++i) {
test_utils->sts_target()->Dispatch(
WrapRunnableRet(peer_->streams_[i], &NrIceMediaStream::ParseAttributes,
candidates_[streams_[i]->name()], &res), NS_DISPATCH_SYNC);
WrapRunnableRet(&res, peer_->streams_[i], &NrIceMediaStream::ParseAttributes,
candidates_[streams_[i]->name()]), NS_DISPATCH_SYNC);
ASSERT_TRUE(NS_SUCCEEDED(res));
}
// Start checks on the other peer.
test_utils->sts_target()->Dispatch(
WrapRunnableRet(peer_->ice_ctx_, &NrIceCtx::StartChecks, &res),
WrapRunnableRet(&res, peer_->ice_ctx_, &NrIceCtx::StartChecks),
NS_DISPATCH_SYNC);
ASSERT_TRUE(NS_SUCCEEDED(res));
}
@@ -708,7 +708,7 @@ class TransportTestPeer : public sigslot::has_slots<> {
TransportResult SendPacket(const unsigned char* data, size_t len) {
TransportResult ret;
test_utils->sts_target()->Dispatch(
WrapRunnableRet(flow_, &TransportFlow::SendPacket, data, len, &ret),
WrapRunnableRet(&ret, flow_, &TransportFlow::SendPacket, data, len),
NS_DISPATCH_SYNC);
return ret;
@@ -755,7 +755,7 @@ class TransportTestPeer : public sigslot::has_slots<> {
TransportLayer::State tstate;
RUN_ON_THREAD(test_utils->sts_target(),
WrapRunnableRet(flow_, &TransportFlow::state, &tstate));
WrapRunnableRet(&tstate, flow_, &TransportFlow::state));
return tstate;
}
@@ -774,8 +774,8 @@ class TransportTestPeer : public sigslot::has_slots<> {
nsresult rv;
uint16_t cipher;
RUN_ON_THREAD(test_utils->sts_target(),
WrapRunnableRet(dtls_, &TransportLayerDtls::GetCipherSuite,
&cipher, &rv));
WrapRunnableRet(&rv, dtls_, &TransportLayerDtls::GetCipherSuite,
&cipher));
if (NS_FAILED(rv)) {
return TLS_NULL_WITH_NULL_NULL; // i.e., not good
@@ -787,8 +787,8 @@ class TransportTestPeer : public sigslot::has_slots<> {
nsresult rv;
uint16_t cipher;
RUN_ON_THREAD(test_utils->sts_target(),
WrapRunnableRet(dtls_, &TransportLayerDtls::GetSrtpCipher,
&cipher, &rv));
WrapRunnableRet(&rv, dtls_, &TransportLayerDtls::GetSrtpCipher,
&cipher));
if (NS_FAILED(rv)) {
return 0; // the SRTP equivalent of TLS_NULL_WITH_NULL_NULL
}
media/mtransport/test/turn_unittest.cpp
+1 -1
View File
@@ -117,7 +117,7 @@ class TurnClient : public ::testing::Test {
r = nr_ip4_port_to_transport_addr(0, 0, protocol_, &addr);
ASSERT_EQ(0, r);
r = nr_socket_local_create(&addr, &real_socket_);
r = nr_socket_local_create(nullptr, &addr, &real_socket_);
ASSERT_EQ(0, r);
if (protocol_ == IPPROTO_TCP) {
media/mtransport/test_nr_socket.cpp
+763
View File
@@ -0,0 +1,763 @@
/* -*- Mode: C++; tab-width: 8; indent-tabs-mode: nil; c-basic-offset: 2 -*- */
/* vim: set ts=2 et sw=2 tw=80: */
/* This Source Code Form is subject to the terms of the Mozilla Public
* License, v. 2.0. If a copy of the MPL was not distributed with this file,
* You can obtain one at http://mozilla.org/MPL/2.0/. */
/*
*/
/*
Based partially on original code from nICEr and nrappkit.
nICEr copyright:
Copyright (c) 2007, Adobe Systems, Incorporated
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are
met:
* Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
* Neither the name of Adobe Systems, Network Resonance nor the names of its
contributors may be used to endorse or promote products derived from
this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
nrappkit copyright:
Copyright (C) 2001-2003, Network Resonance, Inc.
Copyright (C) 2006, Network Resonance, Inc.
All Rights Reserved
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. Neither the name of Network Resonance, Inc. nor the name of any
contributors to this software may be used to endorse or promote
products derived from this software without specific prior written
permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS ``AS IS''
AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
POSSIBILITY OF SUCH DAMAGE.
ekr@rtfm.com Thu Dec 20 20:14:49 2001
*/
// Original author: bcampen@mozilla.com [:bwc]
extern "C" {
#include "stun_msg.h" // for NR_STUN_MAX_MESSAGE_SIZE
#include "nr_api.h"
#include "async_wait.h"
#include "nr_socket.h"
#include "nr_socket_local.h"
#include "stun_hint.h"
#include "transport_addr.h"
}
#include "mozilla/RefPtr.h"
#include "test_nr_socket.h"
#include "runnable_utils.h"
namespace mozilla {
static int test_nat_socket_create(void *obj,
nr_transport_addr *addr,
nr_socket **sockp) {
RefPtr<NrSocketBase> sock = new TestNrSocket(static_cast<TestNat*>(obj));
int r, _status;
r = sock->create(addr);
if (r)
ABORT(r);
r = nr_socket_create_int(static_cast<void *>(sock),
sock->vtbl(), sockp);
if (r)
ABORT(r);
_status = 0;
{
// We will release this reference in destroy(), not exactly the normal
// ownership model, but it is what it is.
NrSocketBase *dummy = sock.forget().take();
(void)dummy;
}
abort:
return _status;
}
static int test_nat_socket_factory_destroy(void **obj) {
TestNat *nat = static_cast<TestNat*>(*obj);
*obj = nullptr;
nat->Release();
return 0;
}
static nr_socket_factory_vtbl test_nat_socket_factory_vtbl = {
test_nat_socket_create,
test_nat_socket_factory_destroy
};
bool TestNat::has_port_mappings() const {
for (TestNrSocket *sock : sockets_) {
if (sock->has_port_mappings()) {
return true;
}
}
return false;
}
bool TestNat::is_my_external_tuple(const nr_transport_addr &addr) const {
for (TestNrSocket *sock : sockets_) {
if (sock->is_my_external_tuple(addr)) {
return true;
}
}
return false;
}
bool TestNat::is_an_internal_tuple(const nr_transport_addr &addr) const {
for (TestNrSocket *sock : sockets_) {
nr_transport_addr addr_behind_nat;
if (sock->getaddr(&addr_behind_nat)) {
MOZ_CRASH("TestNrSocket::getaddr failed!");
}
// TODO(bug 1170299): Remove const_cast when no longer necessary
if (!nr_transport_addr_cmp(const_cast<nr_transport_addr*>(&addr),
&addr_behind_nat,
NR_TRANSPORT_ADDR_CMP_MODE_ALL)) {
return true;
}
}
return false;
}
int TestNat::create_socket_factory(nr_socket_factory **factorypp) {
int r = nr_socket_factory_create_int(this,
&test_nat_socket_factory_vtbl,
factorypp);
if (!r) {
AddRef();
}
return r;
}
TestNrSocket::TestNrSocket(TestNat *nat)
: nat_(nat) {
nat_->insert_socket(this);
}
TestNrSocket::~TestNrSocket() {
nat_->erase_socket(this);
}
nsRefPtr<NrSocket> TestNrSocket::create_external_socket(
const nr_transport_addr &dest_addr) const {
MOZ_ASSERT(nat_->enabled_);
MOZ_ASSERT(!nat_->is_an_internal_tuple(dest_addr));
int r;
nr_transport_addr nat_external_addr;
// Open the socket on an arbitrary port, on the same address.
// TODO(bug 1170299): Remove const_cast when no longer necessary
if ((r = nr_transport_addr_copy(&nat_external_addr,
const_cast<nr_transport_addr*>(&my_addr_)))) {
r_log(LOG_GENERIC,LOG_CRIT, "%s: Failure in nr_transport_addr_copy: %d",
__FUNCTION__, r);
return nullptr;
}
if ((r = nr_transport_addr_set_port(&nat_external_addr, 0))) {
r_log(LOG_GENERIC,LOG_CRIT, "%s: Failure in nr_transport_addr_set_port: %d",
__FUNCTION__, r);
return nullptr;
}
nsRefPtr<NrSocket> external_socket = new NrSocket;
if ((r = external_socket->create(&nat_external_addr))) {
r_log(LOG_GENERIC,LOG_CRIT, "%s: Failure in NrSocket::create: %d",
__FUNCTION__, r);
return nullptr;
}
return external_socket;
}
int TestNrSocket::sendto(const void *msg, size_t len,
int flags, nr_transport_addr *to) {
MOZ_ASSERT(my_addr_.protocol != IPPROTO_TCP);
ASSERT_ON_THREAD(ststhread_);
if (!nat_->enabled_ || nat_->is_an_internal_tuple(*to)) {
return NrSocket::sendto(msg, len, flags, to);
}
destroy_stale_port_mappings();
if (to->protocol == IPPROTO_UDP && nat_->block_udp_) {
// Silently eat the packet
return 0;
}
// Choose our port mapping based on our most selective criteria
PortMapping *port_mapping = get_port_mapping(*to,
std::max(nat_->filtering_type_,
nat_->mapping_type_));
if (!port_mapping) {
// See if we have already made the external socket we need to use.
PortMapping *similar_port_mapping =
get_port_mapping(*to, nat_->mapping_type_);
nsRefPtr<NrSocket> external_socket;
if (similar_port_mapping) {
external_socket = similar_port_mapping->external_socket_;
} else {
external_socket = create_external_socket(*to);
if (!external_socket) {
MOZ_ASSERT(false);
return R_INTERNAL;
}
}
port_mapping = create_port_mapping(*to, external_socket);
port_mappings_.push_back(port_mapping);
if (poll_flags() & PR_POLL_READ) {
// Make sure the new port mapping is ready to receive traffic if the
// TestNrSocket is already waiting.
port_mapping->async_wait(NR_ASYNC_WAIT_READ,
port_mapping_readable_callback,
this,
(char*)__FUNCTION__,
__LINE__);
}
}
// We probably don't want to propagate the flags, since this is a simulated
// external IP address.
return port_mapping->sendto(msg, len, *to);
}
int TestNrSocket::recvfrom(void *buf, size_t maxlen,
size_t *len, int flags,
nr_transport_addr *from) {
MOZ_ASSERT(my_addr_.protocol != IPPROTO_TCP);
ASSERT_ON_THREAD(ststhread_);
int r;
bool ingress_allowed = false;
if (readable_socket_) {
// If any of the external sockets got data, see if it will be passed through
r = readable_socket_->recvfrom(buf, maxlen, len, 0, from);
readable_socket_ = nullptr;
if (!r) {
PortMapping *port_mapping_used;
ingress_allowed = allow_ingress(*from, &port_mapping_used);
if (ingress_allowed && nat_->refresh_on_ingress_ && port_mapping_used) {
port_mapping_used->last_used_ = PR_IntervalNow();
}
}
} else {
// If no external socket has data, see if there's any data that was sent
// directly to the TestNrSocket, and eat it if it isn't supposed to get
// through.
r = NrSocket::recvfrom(buf, maxlen, len, flags, from);
if (!r) {
// We do not use allow_ingress() here because that only handles traffic
// landing on an external port.
ingress_allowed = (!nat_->enabled_ ||
nat_->is_an_internal_tuple(*from));
if (!ingress_allowed) {
r_log(LOG_GENERIC, LOG_INFO, "TestNrSocket %s denying ingress from %s: "
"Not behind the same NAT",
my_addr_.as_string,
from->as_string);
}
}
}
// Kinda lame that we are forced to give the app a readable callback and then
// say "Oh, never mind...", but the alternative is to totally decouple the
// callbacks from STS and the callbacks the app sets. On the bright side, this
// speeds up unit tests where we are verifying that ingress is forbidden,
// since they'll get a readable callback and then an error, instead of having
// to wait for a timeout.
if (!ingress_allowed) {
*len = 0;
r = R_WOULDBLOCK;
}
return r;
}
bool TestNrSocket::allow_ingress(const nr_transport_addr &from,
PortMapping **port_mapping_used) const {
*port_mapping_used = nullptr;
if (!nat_->enabled_)
return true;
if (nat_->is_an_internal_tuple(from))
return true;
*port_mapping_used = get_port_mapping(from, nat_->filtering_type_);
if (!(*port_mapping_used)) {
r_log(LOG_GENERIC, LOG_INFO, "TestNrSocket %s denying ingress from %s: "
"Filtered",
my_addr_.as_string,
from.as_string);
return false;
}
if (is_port_mapping_stale(**port_mapping_used)) {
r_log(LOG_GENERIC, LOG_INFO, "TestNrSocket %s denying ingress from %s: "
"Stale port mapping",
my_addr_.as_string,
from.as_string);
return false;
}
if (!nat_->allow_hairpinning_ && nat_->is_my_external_tuple(from)) {
r_log(LOG_GENERIC, LOG_INFO, "TestNrSocket %s denying ingress from %s: "
"Hairpinning disallowed",
my_addr_.as_string,
from.as_string);
return false;
}
return true;
}
int TestNrSocket::connect(nr_transport_addr *addr) {
ASSERT_ON_THREAD(ststhread_);
if (connect_invoked_ || !port_mappings_.empty()) {
MOZ_CRASH("TestNrSocket::connect() called more than once!");
return R_INTERNAL;
}
if (!nat_->enabled_ || nat_->is_an_internal_tuple(*addr)) {
// This will set connect_invoked_
return NrSocket::connect(addr);
}
nsRefPtr<NrSocket> external_socket(create_external_socket(*addr));
if (!external_socket) {
return R_INTERNAL;
}
PortMapping *port_mapping = create_port_mapping(*addr, external_socket);
port_mappings_.push_back(port_mapping);
port_mapping->external_socket_->connect(addr);
port_mapping->last_used_ = PR_IntervalNow();
if (poll_flags() & PR_POLL_READ) {
port_mapping->async_wait(NR_ASYNC_WAIT_READ,
port_mapping_tcp_passthrough_callback,
this,
(char*)__FUNCTION__,
__LINE__);
}
return 0;
}
int TestNrSocket::write(const void *msg, size_t len, size_t *written) {
ASSERT_ON_THREAD(ststhread_);
if (port_mappings_.empty()) {
// The no-nat case, just pass call through.
r_log(LOG_GENERIC, LOG_INFO, "TestNrSocket %s writing",
my_addr().as_string);
return NrSocket::write(msg, len, written);
} else {
// This is TCP only
MOZ_ASSERT(port_mappings_.size() == 1);
r_log(LOG_GENERIC, LOG_INFO,
"PortMapping %s -> %s writing",
port_mappings_.front()->external_socket_->my_addr().as_string,
port_mappings_.front()->remote_address_.as_string);
return port_mappings_.front()->external_socket_->write(msg, len, written);
}
}
int TestNrSocket::read(void *buf, size_t maxlen, size_t *len) {
ASSERT_ON_THREAD(ststhread_);
if (port_mappings_.empty()) {
return NrSocket::read(buf, maxlen, len);
} else {
MOZ_ASSERT(port_mappings_.size() == 1);
return port_mappings_.front()->external_socket_->read(buf, maxlen, len);
}
}
int TestNrSocket::async_wait(int how, NR_async_cb cb, void *cb_arg,
char *function, int line) {
ASSERT_ON_THREAD(ststhread_);
// Make sure we're waiting on the socket for the internal address
int r = NrSocket::async_wait(how, cb, cb_arg, function, line);
if (r) {
return r;
}
r_log(LOG_GENERIC, LOG_DEBUG, "TestNrSocket %s waiting for %s",
my_addr_.as_string,
how == NR_ASYNC_WAIT_READ ? "read" : "write");
if (is_tcp_connection_behind_nat()) {
// Bypass all port-mapping related logic
return 0;
}
if (my_addr_.protocol == IPPROTO_TCP) {
// For a TCP connection through a simulated NAT, these signals are
// just passed through.
MOZ_ASSERT(port_mappings_.size() == 1);
return port_mappings_.front()->async_wait(
how,
port_mapping_tcp_passthrough_callback,
this,
function,
line);
} else if (how == NR_ASYNC_WAIT_READ) {
// For UDP port mappings, we decouple the writeable callbacks
for (PortMapping *port_mapping : port_mappings_) {
// Be ready to receive traffic on our port mappings
r = port_mapping->async_wait(how,
port_mapping_readable_callback,
this,
function,
line);
if (r) {
return r;
}
}
}
return 0;
}
void TestNrSocket::cancel_port_mapping_async_wait(int how) {
for (PortMapping *port_mapping : port_mappings_) {
port_mapping->cancel(how);
}
}
int TestNrSocket::cancel(int how) {
ASSERT_ON_THREAD(ststhread_);
r_log(LOG_GENERIC, LOG_DEBUG, "TestNrSocket %s stop waiting for %s",
my_addr_.as_string,
how == NR_ASYNC_WAIT_READ ? "read" : "write");
// Writable callbacks are decoupled except for the TCP case
if (how == NR_ASYNC_WAIT_READ || my_addr_.protocol == IPPROTO_TCP) {
cancel_port_mapping_async_wait(how);
}
return NrSocket::cancel(how);
}
bool TestNrSocket::has_port_mappings() const {
return !port_mappings_.empty();
}
bool TestNrSocket::is_my_external_tuple(const nr_transport_addr &addr) const {
for (PortMapping *port_mapping : port_mappings_) {
nr_transport_addr port_mapping_addr;
if (port_mapping->external_socket_->getaddr(&port_mapping_addr)) {
MOZ_CRASH("NrSocket::getaddr failed!");
}
// TODO(bug 1170299): Remove const_cast when no longer necessary
if (!nr_transport_addr_cmp(const_cast<nr_transport_addr*>(&addr),
&port_mapping_addr,
NR_TRANSPORT_ADDR_CMP_MODE_ALL)) {
return true;
}
}
return false;
}
bool TestNrSocket::is_port_mapping_stale(
const PortMapping &port_mapping) const {
PRIntervalTime now = PR_IntervalNow();
PRIntervalTime elapsed_ticks = now - port_mapping.last_used_;
uint32_t idle_duration = PR_IntervalToMilliseconds(elapsed_ticks);
return idle_duration > nat_->mapping_timeout_;
}
void TestNrSocket::destroy_stale_port_mappings() {
for (auto i = port_mappings_.begin(); i != port_mappings_.end();) {
auto temp = i;
++i;
if (is_port_mapping_stale(**temp)) {
r_log(LOG_GENERIC, LOG_INFO,
"TestNrSocket %s destroying port mapping %s -> %s",
my_addr_.as_string,
(*temp)->external_socket_->my_addr().as_string,
(*temp)->remote_address_.as_string);
port_mappings_.erase(temp);
}
}
}
void TestNrSocket::port_mapping_readable_callback(void *ext_sock_v,
int how,
void *test_sock_v) {
TestNrSocket *test_socket = static_cast<TestNrSocket*>(test_sock_v);
NrSocket *external_socket = static_cast<NrSocket*>(ext_sock_v);
test_socket->on_port_mapping_readable(external_socket);
}
void TestNrSocket::on_port_mapping_readable(NrSocket *external_socket) {
if (!readable_socket_) {
readable_socket_ = external_socket;
}
// None of our port mappings should be waiting for readable callbacks
// if nobody is waiting for readable callbacks from us.
MOZ_ASSERT(poll_flags() & PR_POLL_READ);
fire_readable_callback();
}
void TestNrSocket::fire_readable_callback() {
MOZ_ASSERT(poll_flags() & PR_POLL_READ);
// Stop listening on all mapped sockets; we will start listening again
// if the app starts listening to us again.
cancel_port_mapping_async_wait(NR_ASYNC_WAIT_READ);
r_log(LOG_GENERIC, LOG_DEBUG, "TestNrSocket %s ready for read",
my_addr_.as_string);
fire_callback(NR_ASYNC_WAIT_READ);
}
void TestNrSocket::port_mapping_writeable_callback(void *ext_sock_v,
int how,
void *test_sock_v) {
TestNrSocket *test_socket = static_cast<TestNrSocket*>(test_sock_v);
NrSocket *external_socket = static_cast<NrSocket*>(ext_sock_v);
test_socket->write_to_port_mapping(external_socket);
}
void TestNrSocket::write_to_port_mapping(NrSocket *external_socket) {
MOZ_ASSERT(my_addr_.protocol != IPPROTO_TCP);
int r = 0;
for (PortMapping *port_mapping : port_mappings_) {
if (port_mapping->external_socket_ == external_socket) {
// If the send succeeds, or if there was nothing to send, we keep going
r = port_mapping->send_from_queue();
if (r) {
break;
}
}
}
if (r == R_WOULDBLOCK) {
// Re-register for writeable callbacks, since we still have stuff to send
NR_ASYNC_WAIT(external_socket,
NR_ASYNC_WAIT_WRITE,
&TestNrSocket::port_mapping_writeable_callback,
this);
}
}
void TestNrSocket::port_mapping_tcp_passthrough_callback(void *ext_sock_v,
int how,
void *test_sock_v) {
TestNrSocket *test_socket = static_cast<TestNrSocket*>(test_sock_v);
r_log(LOG_GENERIC, LOG_INFO,
"TestNrSocket %s firing %s callback",
test_socket->my_addr().as_string,
how == NR_ASYNC_WAIT_READ ? "readable" : "writeable");
test_socket->fire_callback(how);
}
bool TestNrSocket::is_tcp_connection_behind_nat() const {
return my_addr_.protocol == IPPROTO_TCP && port_mappings_.empty();
}
TestNrSocket::PortMapping* TestNrSocket::get_port_mapping(
const nr_transport_addr &remote_address,
TestNat::NatBehavior filter) const {
int compare_flags;
switch (filter) {
case TestNat::ENDPOINT_INDEPENDENT:
compare_flags = NR_TRANSPORT_ADDR_CMP_MODE_PROTOCOL;
break;
case TestNat::ADDRESS_DEPENDENT:
compare_flags = NR_TRANSPORT_ADDR_CMP_MODE_ADDR;
break;
case TestNat::PORT_DEPENDENT:
compare_flags = NR_TRANSPORT_ADDR_CMP_MODE_ALL;
break;
}
for (PortMapping *port_mapping : port_mappings_) {
// TODO(bug 1170299): Remove const_cast when no longer necessary
if (!nr_transport_addr_cmp(const_cast<nr_transport_addr*>(&remote_address),
&port_mapping->remote_address_,
compare_flags))
return port_mapping;
}
return nullptr;
}
TestNrSocket::PortMapping* TestNrSocket::create_port_mapping(
const nr_transport_addr &remote_address,
const nsRefPtr<NrSocket> &external_socket) const {
r_log(LOG_GENERIC, LOG_INFO, "TestNrSocket %s creating port mapping %s -> %s",
my_addr_.as_string,
external_socket->my_addr().as_string,
remote_address.as_string);
return new PortMapping(remote_address, external_socket);
}
TestNrSocket::PortMapping::PortMapping(
const nr_transport_addr &remote_address,
const nsRefPtr<NrSocket> &external_socket) :
external_socket_(external_socket) {
// TODO(bug 1170299): Remove const_cast when no longer necessary
nr_transport_addr_copy(&remote_address_,
const_cast<nr_transport_addr*>(&remote_address));
}
int TestNrSocket::PortMapping::send_from_queue() {
MOZ_ASSERT(remote_address_.protocol != IPPROTO_TCP);
int r = 0;
while (!send_queue_.empty()) {
UdpPacket &packet = *send_queue_.front();
r_log(LOG_GENERIC, LOG_INFO,
"PortMapping %s -> %s sending from queue to %s",
external_socket_->my_addr().as_string,
remote_address_.as_string,
packet.remote_address_.as_string);
r = external_socket_->sendto(packet.buffer_->data(),
packet.buffer_->len(),
0,
&packet.remote_address_);
if (r) {
if (r != R_WOULDBLOCK) {
r_log(LOG_GENERIC, LOG_ERR, "%s: Fatal error %d, stop trying",
__FUNCTION__, r);
send_queue_.clear();
} else {
r_log(LOG_GENERIC, LOG_INFO, "Would block, will retry later");
}
break;
}
send_queue_.pop_front();
}
return r;
}
int TestNrSocket::PortMapping::sendto(const void *msg,
size_t len,
const nr_transport_addr &to) {
MOZ_ASSERT(remote_address_.protocol != IPPROTO_TCP);
r_log(LOG_GENERIC, LOG_INFO,
"PortMapping %s -> %s sending to %s",
external_socket_->my_addr().as_string,
remote_address_.as_string,
to.as_string);
last_used_ = PR_IntervalNow();
int r = external_socket_->sendto(msg, len, 0,
// TODO(bug 1170299): Remove const_cast when no longer necessary
const_cast<nr_transport_addr*>(&to));
if (r == R_WOULDBLOCK) {
r_log(LOG_GENERIC, LOG_INFO, "Enqueueing UDP packet to %s", to.as_string);
send_queue_.push_back(nsRefPtr<UdpPacket>(new UdpPacket(msg, len, to)));
return 0;
} else if (r) {
r_log(LOG_GENERIC,LOG_ERR, "Error: %d", r);
}
return r;
}
int TestNrSocket::PortMapping::async_wait(int how, NR_async_cb cb, void *cb_arg,
char *function, int line) {
r_log(LOG_GENERIC, LOG_DEBUG,
"PortMapping %s -> %s waiting for %s",
external_socket_->my_addr().as_string,
remote_address_.as_string,
how == NR_ASYNC_WAIT_READ ? "read" : "write");
return external_socket_->async_wait(how, cb, cb_arg, function, line);
}
int TestNrSocket::PortMapping::cancel(int how) {
r_log(LOG_GENERIC, LOG_DEBUG,
"PortMapping %s -> %s stop waiting for %s",
external_socket_->my_addr().as_string,
remote_address_.as_string,
how == NR_ASYNC_WAIT_READ ? "read" : "write");
return external_socket_->cancel(how);
}
} // namespace mozilla
media/mtransport/test_nr_socket.h
+283
View File
@@ -0,0 +1,283 @@
/* -*- Mode: C++; tab-width: 8; indent-tabs-mode: nil; c-basic-offset: 2 -*- */
/* vim: set ts=2 et sw=2 tw=80: */
/* This Source Code Form is subject to the terms of the Mozilla Public
* License, v. 2.0. If a copy of the MPL was not distributed with this file,
* You can obtain one at http://mozilla.org/MPL/2.0/. */
/*
*/
/*
Based partially on original code from nICEr and nrappkit.
nICEr copyright:
Copyright (c) 2007, Adobe Systems, Incorporated
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are
met:
* Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
* Neither the name of Adobe Systems, Network Resonance nor the names of its
contributors may be used to endorse or promote products derived from
this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
nrappkit copyright:
Copyright (C) 2001-2003, Network Resonance, Inc.
Copyright (C) 2006, Network Resonance, Inc.
All Rights Reserved
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. Neither the name of Network Resonance, Inc. nor the name of any
contributors to this software may be used to endorse or promote
products derived from this software without specific prior written
permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS ``AS IS''
AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
POSSIBILITY OF SUCH DAMAGE.
ekr@rtfm.com Thu Dec 20 20:14:49 2001
*/
// Original author: bcampen@mozilla.com [:bwc]
#ifndef test_nr_socket__
#define test_nr_socket__
#include "nr_socket_prsock.h"
extern "C" {
#include "nr_socket.h"
}
#include <set>
#include <vector>
#include <map>
#include <list>
#include "mozilla/UniquePtr.h"
#include "prinrval.h"
namespace mozilla {
class TestNrSocket;
/**
* A group of TestNrSockets that behave as if they were behind the same NAT.
* @note We deliberately avoid addref/release of TestNrSocket here to avoid
* masking lifetime errors elsewhere.
*/
class TestNat {
public:
typedef enum {
/** For mapping, one port is used for all destinations.
* For filtering, allow any external address/port. */
ENDPOINT_INDEPENDENT,
/** For mapping, one port for each destination address (for any port).
* For filtering, allow incoming traffic from addresses that outgoing
* traffic has been sent to. */
ADDRESS_DEPENDENT,
/** For mapping, one port for each destination address/port.
* For filtering, allow incoming traffic only from addresses/ports that
* outgoing traffic has been sent to. */
PORT_DEPENDENT,
} NatBehavior;
TestNat() :
enabled_(false),
filtering_type_(ENDPOINT_INDEPENDENT),
mapping_type_(ENDPOINT_INDEPENDENT),
mapping_timeout_(30000),
allow_hairpinning_(false),
refresh_on_ingress_(false),
block_udp_(false),
sockets_() {}
bool has_port_mappings() const;
// Helps determine whether we're hairpinning
bool is_my_external_tuple(const nr_transport_addr &addr) const;
bool is_an_internal_tuple(const nr_transport_addr &addr) const;
int create_socket_factory(nr_socket_factory **factorypp);
void insert_socket(TestNrSocket *socket) {
sockets_.insert(socket);
}
void erase_socket(TestNrSocket *socket) {
sockets_.erase(socket);
}
NS_INLINE_DECL_THREADSAFE_REFCOUNTING(TestNat);
bool enabled_;
TestNat::NatBehavior filtering_type_;
TestNat::NatBehavior mapping_type_;
uint32_t mapping_timeout_;
bool allow_hairpinning_;
bool refresh_on_ingress_;
bool block_udp_;
private:
std::set<TestNrSocket*> sockets_;
~TestNat(){}
};
/**
* Subclass of NrSocket that can simulate things like being behind a NAT, packet
* loss, latency, packet rewriting, etc. Also exposes some stuff that assists in
* diagnostics.
*/
class TestNrSocket : public NrSocket {
public:
explicit TestNrSocket(TestNat *nat);
virtual ~TestNrSocket();
bool has_port_mappings() const;
bool is_my_external_tuple(const nr_transport_addr &addr) const;
// Overrides of NrSocket
int sendto(const void *msg, size_t len,
int flags, nr_transport_addr *to) override;
int recvfrom(void * buf, size_t maxlen,
size_t *len, int flags,
nr_transport_addr *from) override;
int connect(nr_transport_addr *addr) override;
int write(const void *msg, size_t len, size_t *written) override;
int read(void *buf, size_t maxlen, size_t *len) override;
int async_wait(int how, NR_async_cb cb, void *cb_arg,
char *function, int line) override;
int cancel(int how) override;
private:
class UdpPacket {
public:
UdpPacket(const void *msg, size_t len, const nr_transport_addr &addr) :
buffer_(new DataBuffer(static_cast<const uint8_t*>(msg), len)) {
// TODO(bug 1170299): Remove const_cast when no longer necessary
nr_transport_addr_copy(&remote_address_,
const_cast<nr_transport_addr*>(&addr));
}
nr_transport_addr remote_address_;
UniquePtr<DataBuffer> buffer_;
NS_INLINE_DECL_THREADSAFE_REFCOUNTING(UdpPacket);
private:
~UdpPacket(){}
};
class PortMapping {
public:
PortMapping(const nr_transport_addr &remote_address,
const nsRefPtr<NrSocket> &external_socket);
int sendto(const void *msg, size_t len, const nr_transport_addr &to);
int async_wait(int how, NR_async_cb cb, void *cb_arg,
char *function, int line);
int cancel(int how);
int send_from_queue();
NS_INLINE_DECL_THREADSAFE_REFCOUNTING(PortMapping);
PRIntervalTime last_used_;
nsRefPtr<NrSocket> external_socket_;
// For non-symmetric, most of the data here doesn't matter
nr_transport_addr remote_address_;
private:
~PortMapping(){}
// If external_socket_ returns E_WOULDBLOCK, we don't want to propagate
// that to the code using the TestNrSocket. We can also perhaps use this
// to help simulate things like latency.
std::list<nsRefPtr<UdpPacket>> send_queue_;
};
bool is_port_mapping_stale(const PortMapping &port_mapping) const;
bool allow_ingress(const nr_transport_addr &from,
PortMapping **port_mapping_used) const;
void destroy_stale_port_mappings();
static void port_mapping_readable_callback(void *ext_sock_v,
int how,
void *test_sock_v);
void on_port_mapping_readable(NrSocket *external_socket);
void fire_readable_callback();
static void port_mapping_tcp_passthrough_callback(void *ext_sock_v,
int how,
void *test_sock_v);
void cancel_port_mapping_async_wait(int how);
static void port_mapping_writeable_callback(void *ext_sock_v,
int how,
void *test_sock_v);
void write_to_port_mapping(NrSocket *external_socket);
bool is_tcp_connection_behind_nat() const;
PortMapping* get_port_mapping(const nr_transport_addr &remote_addr,
TestNat::NatBehavior filter) const;
PortMapping* create_port_mapping(
const nr_transport_addr &remote_addr,
const nsRefPtr<NrSocket> &external_socket) const;
nsRefPtr<NrSocket> create_external_socket(
const nr_transport_addr &remote_addr) const;
nsRefPtr<NrSocket> readable_socket_;
nsRefPtr<TestNat> nat_;
// Since our comparison logic is different depending on what kind of NAT
// we simulate, and the STL does not make it very easy to switch out the
// comparison function at runtime, and these lists are going to be very
// small anyway, we just brute-force it.
std::list<nsRefPtr<PortMapping>> port_mappings_;
};
} // namespace mozilla
#endif // test_nr_socket__
media/mtransport/third_party/nICEr/src/ice/ice_component.c
Vendored
+2 -2
View File
@@ -196,7 +196,7 @@ static int nr_ice_component_initialize_udp(struct nr_ice_ctx_ *ctx,nr_ice_compon
continue;
}
r_log(LOG_ICE,LOG_DEBUG,"ICE(%s): host address %s",ctx->label,addrs[i].addr.as_string);
if(r=nr_socket_local_create(&addrs[i].addr,&sock)){
if((r=nr_socket_factory_create_socket(ctx->socket_factory,&addrs[i].addr,&sock))){
r_log(LOG_ICE,LOG_WARNING,"ICE(%s): couldn't create socket for address %s",ctx->label,addrs[i].addr.as_string);
continue;
}
@@ -323,7 +323,7 @@ static int nr_ice_component_initialize_tcp(struct nr_ice_ctx_ *ctx,nr_ice_compon
addr.protocol = IPPROTO_TCP;
if ((r=nr_transport_addr_fmt_addr_string(&addr)))
ABORT(r);
if((r=nr_socket_local_create(&addr, &sock))){
if((r=nr_socket_factory_create_socket(ctx->socket_factory,&addr,&sock))){
r_log(LOG_ICE,LOG_DEBUG,"ICE(%s): couldn't create socket for address %s",ctx->label,addr.as_string);
continue;
}
media/mtransport/third_party/nICEr/src/ice/ice_ctx.c
Vendored
+19
View File
@@ -54,6 +54,7 @@ static char *RCSSTRING __UNUSED__="$Id: ice_ctx.c,v 1.2 2008/04/28 17:59:01 ekr
#include "nr_crypto.h"
#include "async_timer.h"
#include "util.h"
#include "nr_socket_local.h"
int LOG_ICE = 0;
@@ -65,6 +66,14 @@ static int nr_ice_fetch_turn_servers(int ct, nr_ice_turn_server **out);
#endif /* USE_TURN */
static void nr_ice_ctx_destroy_cb(NR_SOCKET s, int how, void *cb_arg);
static int nr_ice_ctx_pair_new_trickle_candidates(nr_ice_ctx *ctx, nr_ice_candidate *cand);
static int no_op(void **obj) {
return 0;
}
static nr_socket_factory_vtbl default_socket_factory_vtbl = {
nr_socket_local_create,
no_op
};
int nr_ice_fetch_stun_servers(int ct, nr_ice_stun_server **out)
{
@@ -229,6 +238,12 @@ int nr_ice_ctx_set_turn_tcp_socket_wrapper(nr_ice_ctx *ctx, nr_socket_wrapper_fa
return(_status);
}
void nr_ice_ctx_set_socket_factory(nr_ice_ctx *ctx, nr_socket_factory *factory)
{
nr_socket_factory_destroy(&ctx->socket_factory);
ctx->socket_factory = factory;
}
#ifdef USE_TURN
int nr_ice_fetch_turn_servers(int ct, nr_ice_turn_server **out)
{
@@ -381,6 +396,9 @@ int nr_ice_ctx_create(char *label, UINT4 flags, nr_ice_ctx **ctxp)
ctx->Ta = 20;
if (r=nr_socket_factory_create_int(NULL, &default_socket_factory_vtbl, &ctx->socket_factory))
ABORT(r);
STAILQ_INIT(&ctx->streams);
STAILQ_INIT(&ctx->sockets);
STAILQ_INIT(&ctx->foundations);
@@ -439,6 +457,7 @@ static void nr_ice_ctx_destroy_cb(NR_SOCKET s, int how, void *cb_arg)
nr_resolver_destroy(&ctx->resolver);
nr_interface_prioritizer_destroy(&ctx->interface_prioritizer);
nr_socket_wrapper_factory_destroy(&ctx->turn_tcp_socket_wrapper);
nr_socket_factory_destroy(&ctx->socket_factory);
RFREE(ctx);
}
media/mtransport/third_party/nICEr/src/ice/ice_ctx.h
Vendored
+2
View File
@@ -130,6 +130,7 @@ struct nr_ice_ctx_ {
nr_resolver *resolver; /* The resolver to use */
nr_interface_prioritizer *interface_prioritizer; /* Priority decision logic */
nr_socket_wrapper_factory *turn_tcp_socket_wrapper; /* The TURN TCP socket wrapper to use */
nr_socket_factory *socket_factory;
nr_ice_foundation_head foundations;
@@ -173,6 +174,7 @@ int nr_ice_ctx_set_turn_servers(nr_ice_ctx *ctx,nr_ice_turn_server *servers, int
int nr_ice_ctx_set_resolver(nr_ice_ctx *ctx, nr_resolver *resolver);
int nr_ice_ctx_set_interface_prioritizer(nr_ice_ctx *ctx, nr_interface_prioritizer *prioritizer);
int nr_ice_ctx_set_turn_tcp_socket_wrapper(nr_ice_ctx *ctx, nr_socket_wrapper_factory *wrapper);
void nr_ice_ctx_set_socket_factory(nr_ice_ctx *ctx, nr_socket_factory *factory);
int nr_ice_ctx_set_trickle_cb(nr_ice_ctx *ctx, nr_ice_trickle_candidate_cb cb, void *cb_arg);
#define NR_ICE_MAX_ATTRIBUTE_SIZE 256
media/mtransport/third_party/nICEr/src/net/nr_socket.c
Vendored
+40
View File
@@ -135,3 +135,43 @@ int nr_socket_read(nr_socket *sock,void * restrict buf, size_t maxlen,
CHECK_DEFINED(sread);
return sock->vtbl->sread(sock->obj, buf, maxlen, len);
}
int nr_socket_factory_create_int(void *obj,
nr_socket_factory_vtbl *vtbl, nr_socket_factory **factorypp)
{
int _status;
nr_socket_factory *factoryp=0;
if(!(factoryp=RCALLOC(sizeof(nr_socket_factory))))
ABORT(R_NO_MEMORY);
factoryp->obj = obj;
factoryp->vtbl = vtbl;
*factorypp = factoryp;
_status=0;
abort:
return(_status);
}
int nr_socket_factory_destroy(nr_socket_factory **factorypp)
{
nr_socket_factory *factoryp;
if (!factorypp || !*factorypp)
return (0);
factoryp = *factorypp;
*factorypp = NULL;
factoryp->vtbl->destroy(&factoryp->obj);
RFREE(factoryp);
return (0);
}
int nr_socket_factory_create_socket(nr_socket_factory *factory, nr_transport_addr *addr, nr_socket **sockp)
{
return factory->vtbl->create_socket(factory->obj, addr, sockp);
}
media/mtransport/third_party/nICEr/src/net/nr_socket.h
Vendored
+13
View File
@@ -72,6 +72,15 @@ typedef struct nr_socket_ {
nr_socket_vtbl *vtbl;
} nr_socket;
typedef struct nr_socket_factory_vtbl_ {
int (*create_socket)(void *obj, nr_transport_addr *addr, nr_socket **sockp);
int (*destroy)(void **obj);
} nr_socket_factory_vtbl;
typedef struct nr_socket_factory_ {
void *obj;
nr_socket_factory_vtbl *vtbl;
} nr_socket_factory;
/* To be called by constructors */
int nr_socket_create_int(void *obj, nr_socket_vtbl *vtbl, nr_socket **sockp);
@@ -87,5 +96,9 @@ int nr_socket_connect(nr_socket *sock, nr_transport_addr *addr);
int nr_socket_write(nr_socket *sock,const void *msg, size_t len, size_t *written, int flags);
int nr_socket_read(nr_socket *sock, void * restrict buf, size_t maxlen, size_t *len, int flags);
int nr_socket_factory_create_int(void *obj, nr_socket_factory_vtbl *vtbl, nr_socket_factory **factorypp);
int nr_socket_factory_destroy(nr_socket_factory **factoryp);
int nr_socket_factory_create_socket(nr_socket_factory *factory, nr_transport_addr *addr, nr_socket **sockp);
#endif
media/mtransport/third_party/nICEr/src/net/nr_socket_local.h
Vendored
+1 -1
View File
@@ -35,7 +35,7 @@ OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
#ifndef _nr_socket_local_h
#define _nr_socket_local_h
int nr_socket_local_create(nr_transport_addr *addr, nr_socket **sockp);
int nr_socket_local_create(void *obj, nr_transport_addr *addr, nr_socket **sockp);
#endif
media/webrtc/signaling/src/media-conduit/WebrtcGmpVideoCodec.cpp
+6 -9
View File
@@ -282,12 +282,11 @@ WebrtcGmpVideoEncoder::Encode(const webrtc::I420VideoFrame& aInputImage,
int32_t ret;
mozilla::SyncRunnable::DispatchToThread(mGMPThread,
WrapRunnableRet(this,
WrapRunnableRet(&ret, this,
&WebrtcGmpVideoEncoder::Encode_g,
&aInputImage,
aCodecSpecificInfo,
aFrameTypes,
&ret));
aFrameTypes));
return ret;
}
@@ -415,10 +414,9 @@ WebrtcGmpVideoEncoder::SetRates(uint32_t aNewBitRate, uint32_t aFrameRate)
int32_t ret;
MOZ_ASSERT(mGMPThread);
mozilla::SyncRunnable::DispatchToThread(mGMPThread,
WrapRunnableRet(this,
WrapRunnableRet(&ret, this,
&WebrtcGmpVideoEncoder::SetRates_g,
aNewBitRate, aFrameRate,
&ret));
aNewBitRate, aFrameRate));
return WEBRTC_VIDEO_CODEC_OK;
}
@@ -686,14 +684,13 @@ WebrtcGmpVideoDecoder::Decode(const webrtc::EncodedImage& aInputImage,
int32_t ret;
MOZ_ASSERT(mGMPThread);
mozilla::SyncRunnable::DispatchToThread(mGMPThread,
WrapRunnableRet(this,
WrapRunnableRet(&ret, this,
&WebrtcGmpVideoDecoder::Decode_g,
aInputImage,
aMissingFrames,
aFragmentation,
aCodecSpecificInfo,
aRenderTimeMs,
&ret));
aRenderTimeMs));
return ret;
}
media/webrtc/signaling/src/peerconnection/PeerConnectionImpl.cpp
+1 -1
View File
@@ -155,7 +155,7 @@ public:
class WrappableJSErrorResult {
public:
WrappableJSErrorResult() : isCopy(false) {}
WrappableJSErrorResult(WrappableJSErrorResult &other) : mRv(), isCopy(true) {}
WrappableJSErrorResult(const WrappableJSErrorResult &other) : mRv(), isCopy(true) {}
~WrappableJSErrorResult() {
if (isCopy) {
MOZ_ASSERT(NS_IsMainThread());
media/webrtc/signaling/test/common.build
+1 -1
View File
@@ -25,6 +25,7 @@ LOCAL_INCLUDES += [
'/media/mtransport/third_party/nrappkit/src/event',
'/media/mtransport/third_party/nrappkit/src/log',
'/media/mtransport/third_party/nrappkit/src/plugin',
'/media/mtransport/third_party/nrappkit/src/port/generic/include',
'/media/mtransport/third_party/nrappkit/src/registry',
'/media/mtransport/third_party/nrappkit/src/share',
'/media/mtransport/third_party/nrappkit/src/stats',
@@ -74,7 +75,6 @@ if CONFIG['OS_TARGET'] == 'Darwin':
if CONFIG['OS_TARGET'] in ('DragonFly', 'FreeBSD', 'NetBSD', 'OpenBSD'):
LOCAL_INCLUDES += [
'/media/mtransport/third_party/nrappkit/src/port/darwin/include',
'/media/mtransport/third_party/nrappkit/src/port/generic/include',
]
USE_LIBS += [
media/webrtc/signaling/test/mediaconduit_unittests.cpp
+13 -19
View File
@@ -526,16 +526,14 @@ class TransportConduitTest : public ::testing::Test
//get pointer to AudioSessionConduit
int err=0;
mozilla::SyncRunnable::DispatchToThread(gMainThread,
WrapRunnableNMRet(
&mozilla::AudioSessionConduit::Create,
&mAudioSession));
WrapRunnableNMRet(&mAudioSession,
&mozilla::AudioSessionConduit::Create));
if( !mAudioSession )
ASSERT_NE(mAudioSession, (void*)nullptr);
mozilla::SyncRunnable::DispatchToThread(gMainThread,
WrapRunnableNMRet(
&mozilla::AudioSessionConduit::Create,
&mAudioSession2));
WrapRunnableNMRet(&mAudioSession2,
&mozilla::AudioSessionConduit::Create));
if( !mAudioSession2 )
ASSERT_NE(mAudioSession2, (void*)nullptr);
@@ -590,17 +588,15 @@ class TransportConduitTest : public ::testing::Test
int err = 0;
//get pointer to VideoSessionConduit
mozilla::SyncRunnable::DispatchToThread(gMainThread,
WrapRunnableNMRet(
&mozilla::VideoSessionConduit::Create,
&mVideoSession));
WrapRunnableNMRet(&mVideoSession,
&mozilla::VideoSessionConduit::Create));
if( !mVideoSession )
ASSERT_NE(mVideoSession, (void*)nullptr);
// This session is for other one
mozilla::SyncRunnable::DispatchToThread(gMainThread,
WrapRunnableNMRet(
&mozilla::VideoSessionConduit::Create,
&mVideoSession2));
WrapRunnableNMRet(&mVideoSession2,
&mozilla::VideoSessionConduit::Create));
if( !mVideoSession2 )
ASSERT_NE(mVideoSession2,(void*)nullptr);
@@ -688,9 +684,8 @@ class TransportConduitTest : public ::testing::Test
mozilla::RefPtr<mozilla::VideoSessionConduit> videoSession;
//get pointer to VideoSessionConduit
mozilla::SyncRunnable::DispatchToThread(gMainThread,
WrapRunnableNMRet(
&mozilla::VideoSessionConduit::Create,
&videoSession));
WrapRunnableNMRet(&videoSession,
&mozilla::VideoSessionConduit::Create));
if( !videoSession )
ASSERT_NE(videoSession, (void*)nullptr);
@@ -799,9 +794,8 @@ class TransportConduitTest : public ::testing::Test
// Get pointer to VideoSessionConduit.
mozilla::SyncRunnable::DispatchToThread(gMainThread,
WrapRunnableNMRet(
&mozilla::VideoSessionConduit::Create,
&mVideoSession));
WrapRunnableNMRet(&mVideoSession,
&mozilla::VideoSessionConduit::Create));
if( !mVideoSession )
ASSERT_NE(mVideoSession, (void*)nullptr);
@@ -1040,7 +1034,7 @@ int main(int argc, char **argv)
int result;
gGtestThread->Dispatch(
mozilla::WrapRunnableNMRet(gtest_main, argc, argv, &result), NS_DISPATCH_NORMAL);
mozilla::WrapRunnableNMRet(&result, gtest_main, argc, argv), NS_DISPATCH_NORMAL);
// Here we handle the event queue for dispatches to the main thread
// When the GTest thread is complete it will send one more dispatch
media/webrtc/signaling/test/mediapipeline_unittest.cpp
+1 -1
View File
@@ -161,7 +161,7 @@ class TestAgent {
mozilla::SyncRunnable::DispatchToThread(
test_utils->sts_target(),
WrapRunnableRet(audio_->GetStream(), &Fake_MediaStream::Start, &ret));
WrapRunnableRet(&ret, audio_->GetStream(), &Fake_MediaStream::Start));
ASSERT_TRUE(NS_SUCCEEDED(ret));
}
media/webrtc/signaling/test/signaling_unittests.cpp
+36 -42
View File
@@ -662,8 +662,8 @@ class PCDispatchWrapper : public nsSupportsWeakReference
// Instead we are dispatching back to the same method for
// all of these.
gMainThread->Dispatch(
WrapRunnableRet(this, &PCDispatchWrapper::Initialize,
aObserver, aWindow, aConfiguration, aThread, &rv),
WrapRunnableRet(&rv, this, &PCDispatchWrapper::Initialize,
aObserver, aWindow, aConfiguration, aThread),
NS_DISPATCH_SYNC);
rv = NS_OK;
}
@@ -685,8 +685,7 @@ class PCDispatchWrapper : public nsSupportsWeakReference
}
} else {
gMainThread->Dispatch(
WrapRunnableRet(this, &PCDispatchWrapper::CreateOffer,
aOptions, &rv),
WrapRunnableRet(&rv, this, &PCDispatchWrapper::CreateOffer, aOptions),
NS_DISPATCH_SYNC);
}
@@ -700,7 +699,7 @@ class PCDispatchWrapper : public nsSupportsWeakReference
rv = pc_->CreateAnswer();
} else {
gMainThread->Dispatch(
WrapRunnableRet(this, &PCDispatchWrapper::CreateAnswer, &rv),
WrapRunnableRet(&rv, this, &PCDispatchWrapper::CreateAnswer),
NS_DISPATCH_SYNC);
}
@@ -714,8 +713,8 @@ class PCDispatchWrapper : public nsSupportsWeakReference
rv = pc_->SetLocalDescription(aAction, aSDP);
} else {
gMainThread->Dispatch(
WrapRunnableRet(this, &PCDispatchWrapper::SetLocalDescription,
aAction, aSDP, &rv),
WrapRunnableRet(&rv, this, &PCDispatchWrapper::SetLocalDescription,
aAction, aSDP),
NS_DISPATCH_SYNC);
}
@@ -729,8 +728,8 @@ class PCDispatchWrapper : public nsSupportsWeakReference
rv = pc_->SetRemoteDescription(aAction, aSDP);
} else {
gMainThread->Dispatch(
WrapRunnableRet(this, &PCDispatchWrapper::SetRemoteDescription,
aAction, aSDP, &rv),
WrapRunnableRet(&rv, this, &PCDispatchWrapper::SetRemoteDescription,
aAction, aSDP),
NS_DISPATCH_SYNC);
}
@@ -745,8 +744,8 @@ class PCDispatchWrapper : public nsSupportsWeakReference
rv = pc_->AddIceCandidate(aCandidate, aMid, aLevel);
} else {
gMainThread->Dispatch(
WrapRunnableRet(this, &PCDispatchWrapper::AddIceCandidate,
aCandidate, aMid, aLevel, &rv),
WrapRunnableRet(&rv, this, &PCDispatchWrapper::AddIceCandidate,
aCandidate, aMid, aLevel),
NS_DISPATCH_SYNC);
}
return rv;
@@ -761,8 +760,8 @@ class PCDispatchWrapper : public nsSupportsWeakReference
rv = pc_->AddTrack(*aTrack, *aMediaStream);
} else {
gMainThread->Dispatch(
WrapRunnableRet(this, &PCDispatchWrapper::AddTrack, aTrack,
aMediaStream, &rv),
WrapRunnableRet(&rv, this, &PCDispatchWrapper::AddTrack, aTrack,
aMediaStream),
NS_DISPATCH_SYNC);
}
@@ -776,7 +775,7 @@ class PCDispatchWrapper : public nsSupportsWeakReference
rv = pc_->RemoveTrack(*aTrack);
} else {
gMainThread->Dispatch(
WrapRunnableRet(this, &PCDispatchWrapper::RemoveTrack, aTrack, &rv),
WrapRunnableRet(&rv, this, &PCDispatchWrapper::RemoveTrack, aTrack),
NS_DISPATCH_SYNC);
}
@@ -790,8 +789,8 @@ class PCDispatchWrapper : public nsSupportsWeakReference
rv = pc_->GetLocalDescription(aSDP);
} else {
gMainThread->Dispatch(
WrapRunnableRet(this, &PCDispatchWrapper::GetLocalDescription,
aSDP, &rv),
WrapRunnableRet(&rv, this, &PCDispatchWrapper::GetLocalDescription,
aSDP),
NS_DISPATCH_SYNC);
}
@@ -805,8 +804,8 @@ class PCDispatchWrapper : public nsSupportsWeakReference
rv = pc_->GetRemoteDescription(aSDP);
} else {
gMainThread->Dispatch(
WrapRunnableRet(this, &PCDispatchWrapper::GetRemoteDescription,
aSDP, &rv),
WrapRunnableRet(&rv, this, &PCDispatchWrapper::GetRemoteDescription,
aSDP),
NS_DISPATCH_SYNC);
}
@@ -820,8 +819,7 @@ class PCDispatchWrapper : public nsSupportsWeakReference
result = pc_->SignalingState();
} else {
gMainThread->Dispatch(
WrapRunnableRet(this, &PCDispatchWrapper::SignalingState,
&result),
WrapRunnableRet(&result, this, &PCDispatchWrapper::SignalingState),
NS_DISPATCH_SYNC);
}
@@ -835,8 +833,7 @@ class PCDispatchWrapper : public nsSupportsWeakReference
result = pc_->IceConnectionState();
} else {
gMainThread->Dispatch(
WrapRunnableRet(this, &PCDispatchWrapper::IceConnectionState,
&result),
WrapRunnableRet(&result, this, &PCDispatchWrapper::IceConnectionState),
NS_DISPATCH_SYNC);
}
@@ -850,8 +847,7 @@ class PCDispatchWrapper : public nsSupportsWeakReference
result = pc_->IceGatheringState();
} else {
gMainThread->Dispatch(
WrapRunnableRet(this, &PCDispatchWrapper::IceGatheringState,
&result),
WrapRunnableRet(&result, this, &PCDispatchWrapper::IceGatheringState),
NS_DISPATCH_SYNC);
}
@@ -865,7 +861,7 @@ class PCDispatchWrapper : public nsSupportsWeakReference
rv = pc_->Close();
} else {
gMainThread->Dispatch(
WrapRunnableRet(this, &PCDispatchWrapper::Close, &rv),
WrapRunnableRet(&rv, this, &PCDispatchWrapper::Close),
NS_DISPATCH_SYNC);
}
@@ -1067,7 +1063,7 @@ class SignalingAgent {
nsresult ret;
mozilla::SyncRunnable::DispatchToThread(
test_utils->sts_target(),
WrapRunnableRet(audio_stream, &Fake_MediaStream::Start, &ret));
WrapRunnableRet(&ret, audio_stream, &Fake_MediaStream::Start));
ASSERT_TRUE(NS_SUCCEEDED(ret));
stream = audio_stream;
@@ -1127,9 +1123,9 @@ class SignalingAgent {
{
LocalSourceStreamInfo* info;
mozilla::SyncRunnable::DispatchToThread(
gMainThread, WrapRunnableRet(
gMainThread, WrapRunnableRet(&info,
pc->media(), &PeerConnectionMedia::GetLocalStreamById,
streamId, &info));
streamId));
ASSERT_TRUE(info) << "No such local stream id: " << streamId;
@@ -1137,10 +1133,9 @@ class SignalingAgent {
mozilla::SyncRunnable::DispatchToThread(
gMainThread,
WrapRunnableRet(info,
WrapRunnableRet(&pipeline, info,
&SourceStreamInfo::GetPipelineByTrackId_m,
trackId,
&pipeline));
trackId));
ASSERT_TRUE(pipeline) << "No such local track id: " << trackId;
@@ -1175,9 +1170,9 @@ class SignalingAgent {
{
RemoteSourceStreamInfo* info;
mozilla::SyncRunnable::DispatchToThread(
gMainThread, WrapRunnableRet(
gMainThread, WrapRunnableRet(&info,
pc->media(), &PeerConnectionMedia::GetRemoteStreamById,
streamId, &info));
streamId));
ASSERT_TRUE(info) << "No such remote stream id: " << streamId;
@@ -1185,10 +1180,9 @@ class SignalingAgent {
mozilla::SyncRunnable::DispatchToThread(
gMainThread,
WrapRunnableRet(info,
WrapRunnableRet(&pipeline, info,
&SourceStreamInfo::GetPipelineByTrackId_m,
trackId,
&pipeline));
trackId));
ASSERT_TRUE(pipeline) << "No such remote track id: " << trackId;
@@ -1300,7 +1294,7 @@ class SignalingAgent {
nsresult ret;
mozilla::SyncRunnable::DispatchToThread(
test_utils->sts_target(),
WrapRunnableRet(audio_stream, &Fake_MediaStream::Start, &ret));
WrapRunnableRet(&ret, audio_stream, &Fake_MediaStream::Start));
ASSERT_TRUE(NS_SUCCEEDED(ret));
@@ -1509,14 +1503,14 @@ class SignalingAgent {
SourceStreamInfo* streamInfo;
if (local) {
mozilla::SyncRunnable::DispatchToThread(
gMainThread, WrapRunnableRet(
gMainThread, WrapRunnableRet(&streamInfo,
pc->media(), &PeerConnectionMedia::GetLocalStreamByIndex,
stream, &streamInfo));
stream));
} else {
mozilla::SyncRunnable::DispatchToThread(
gMainThread, WrapRunnableRet(
gMainThread, WrapRunnableRet(&streamInfo,
pc->media(), &PeerConnectionMedia::GetRemoteStreamByIndex,
stream, &streamInfo));
stream));
}
if (!streamInfo) {
@@ -4693,7 +4687,7 @@ int main(int argc, char **argv) {
int result;
gGtestThread->Dispatch(
WrapRunnableNMRet(gtest_main, argc, argv, &result), NS_DISPATCH_NORMAL);
WrapRunnableNMRet(&result, gtest_main, argc, argv), NS_DISPATCH_NORMAL);
// Here we handle the event queue for dispatches to the main thread
// When the GTest thread is complete it will send one more dispatch
media/webrtc/signaling/test/standalone/moz.build
+1 -1
View File
@@ -25,7 +25,7 @@ for var in ('MOZILLA_INTERNAL_API', 'MOZILLA_XPCOMRT_API'):
if CONFIG['OS_TARGET'] == 'Linux':
USE_LIBS += [
'static:/nsprpub/lib/libc/src/plc4',
'nspr',
]
OS_LIBS += [
'-lrt',
mfbt/TemplateLib.h
+21
View File
@@ -20,6 +20,8 @@
#include <limits.h>
#include <stddef.h>
#include "mozilla/TypeTraits.h"
namespace mozilla {
namespace tl {
@@ -105,6 +107,25 @@ struct MulOverflowMask
template<> struct MulOverflowMask<0> { /* Error */ };
template<> struct MulOverflowMask<1> { static const size_t value = 0; };
/**
* And<bool...> computes the logical 'and' of its argument booleans.
*
* Examples:
* mozilla::t1::And<true, true>::value is true.
* mozilla::t1::And<true, false>::value is false.
* mozilla::t1::And<>::value is true.
*/
template<bool...>
struct And;
template<>
struct And<> : public TrueType { };
template<bool C1, bool... Cn>
struct And<C1, Cn...>
: public Conditional<C1, And<Cn...>, FalseType>::Type { };
} // namespace tl
} // namespace mozilla
mfbt/Tuple.h
+301
View File
@@ -0,0 +1,301 @@
/* -*- Mode: C++; tab-width: 8; indent-tabs-mode: nil; c-basic-offset: 2 -*- */
/* vim: set ts=8 sts=2 et sw=2 tw=80: */
/* This Source Code Form is subject to the terms of the Mozilla Public
* License, v. 2.0. If a copy of the MPL was not distributed with this
* file, You can obtain one at http://mozilla.org/MPL/2.0/. */
/* A variadic tuple class. */
#ifndef mozilla_Tuple_h
#define mozilla_Tuple_h
#include "mozilla/Move.h"
#include "mozilla/TemplateLib.h"
#include "mozilla/TypeTraits.h"
#include <stddef.h>
namespace mozilla {
namespace detail {
/*
* A helper class that allows passing around multiple variadic argument lists
* by grouping them.
*/
template<typename... Ts>
struct Group;
/*
* CheckConvertibility checks whether each type in a source pack of types
* is convertible to the corresponding type in a target pack of types.
*
* It is intended to be invoked like this:
* CheckConvertibility<Group<SourceTypes...>, Group<TargetTypes...>>
* 'Group' is used to separate types in the two packs (otherwise if we just
* wrote 'CheckConvertibility<SourceTypes..., TargetTypes...', it couldn't
* know where the first pack ends and the second begins).
*
* Note that we need to check explicitly that the two packs are of the same
* size, because attempting to simultaneously expand two parameter packs
* is an error (and it would be a hard error, because it wouldn't be in the
* immediate context of the caller).
*/
template<typename Source, typename Target, bool SameSize>
struct CheckConvertibilityImpl;
template<typename Source, typename Target>
struct CheckConvertibilityImpl<Source, Target, false>
: FalseType {};
template<typename... SourceTypes, typename... TargetTypes>
struct CheckConvertibilityImpl<Group<SourceTypes...>, Group<TargetTypes...>, true>
: IntegralConstant<bool, tl::And<IsConvertible<SourceTypes, TargetTypes>::value...>::value> { };
template<typename Source, typename Target>
struct CheckConvertibility;
template<typename... SourceTypes, typename... TargetTypes>
struct CheckConvertibility<Group<SourceTypes...>, Group<TargetTypes...>>
: CheckConvertibilityImpl<Group<SourceTypes...>, Group<TargetTypes...>,
sizeof...(SourceTypes) == sizeof...(TargetTypes)> { };
/*
* TupleImpl is a helper class used to implement mozilla::Tuple.
* It represents one node in a recursive inheritance hierarchy.
* 'Index' is the 0-based index of the tuple element stored in this node;
* 'Elements...' are the types of the elements stored in this node and its
* base classes.
*
* Example:
* Tuple<int, float, char> inherits from
* TupleImpl<0, int, float, char>, which stores the 'int' and inherits from
* TupleImpl<1, float, char>, which stores the 'float' and inherits from
* TupleImpl<2, char>, which stores the 'char' and inherits from
* TupleImpl<3>, which stores nothing and terminates the recursion.
*
* The purpose of the 'Index' parameter is to allow efficient index-based
* access to a tuple element: given a tuple, and an index 'I' that we wish to
* access, we can cast the tuple to the base which stores the I'th element
* by performing template argument deduction against 'TupleImpl<I, E...>',
* where 'I' is specified explicitly and 'E...' is deduced (this is what the
* non-member 'Get<N>(t)' function does).
*
* This implementation strategy is borrowed from libstdc++'s std::tuple
* implementation.
*/
template<std::size_t Index, typename... Elements>
struct TupleImpl;
/*
* The base case of the inheritance recursion (and also the implementation
* of an empty tuple).
*/
template<std::size_t Index>
struct TupleImpl<Index> {};
/*
* One node of the recursive inheritance hierarchy. It stores the element at
* index 'Index' of a tuple, of type 'HeadT', and inherits from the nodes
* that store the remaining elements, of types 'TailT...'.
*/
template<std::size_t Index, typename HeadT, typename... TailT>
struct TupleImpl<Index, HeadT, TailT...>
: public TupleImpl<Index + 1, TailT...>
{
typedef TupleImpl<Index + 1, TailT...> Base;
// Accessors for the head and the tail.
// These are static, because the intended usage is for the caller to,
// given a tuple, obtain the type B of the base class which stores the
// element of interest, and then call B::Head(tuple) to access it.
// (Tail() is mostly for internal use, but is exposed for consistency.)
static HeadT& Head(TupleImpl& aTuple) { return aTuple.mHead; }
static const HeadT& Head(const TupleImpl& aTuple) { return aTuple.mHead; }
static Base& Tail(TupleImpl& aTuple) { return aTuple; }
static const Base& Tail(const TupleImpl& aTuple) { return aTuple; }
TupleImpl() : Base(), mHead() { }
// Construct from const references to the elements.
explicit TupleImpl(const HeadT& aHead, const TailT&... aTail)
: Base(aTail...), mHead(aHead) { }
// Construct from objects that are convertible to the elements.
// This constructor is enabled only when the argument types are actually
// convertible to the element types, otherwise it could become a better
// match for certain invocations than the copy constructor.
template <typename OtherHeadT, typename... OtherTailT,
typename = typename EnableIf<
CheckConvertibility<
Group<OtherHeadT, OtherTailT...>,
Group<HeadT, TailT...>>::value>::Type>
explicit TupleImpl(OtherHeadT&& aHead, OtherTailT&&... aTail)
: Base(Forward<OtherTailT>(aTail)...), mHead(Forward<OtherHeadT>(aHead)) { }
// Copy and move constructors.
// We'd like to use '= default' to implement these, but MSVC 2013's support
// for '= default' is incomplete and this doesn't work.
TupleImpl(const TupleImpl& aOther)
: Base(Tail(aOther))
, mHead(Head(aOther)) {}
TupleImpl(TupleImpl&& aOther)
: Base(Move(Tail(aOther)))
, mHead(Move(Head(aOther))) {}
// Copy and move assignment operators.
TupleImpl& operator=(const TupleImpl& aOther)
{
Head(*this) = Head(aOther);
Tail(*this) = Tail(aOther);
return *this;
}
TupleImpl& operator=(TupleImpl&& aOther)
{
Head(*this) = Move(Head(aOther));
Tail(*this) = Move(Tail(aOther));
return *this;
}
private:
HeadT mHead; // The element stored at this index in the tuple.
};
} // namespace detail
/**
* Tuple is a class that stores zero or more objects, whose types are specified
* as template parameters. It can be thought of as a generalization of Pair,
* (which can be thought of as a 2-tuple).
*
* Tuple allows index-based access to its elements (with the index having to be
* known at compile time) via the non-member function 'Get<N>(tuple)'.
*/
template<typename... Elements>
class Tuple : public detail::TupleImpl<0, Elements...>
{
typedef detail::TupleImpl<0, Elements...> Impl;
public:
// The constructors and assignment operators here are simple wrappers
// around those in TupleImpl.
Tuple() : Impl() { }
explicit Tuple(const Elements&... aElements) : Impl(aElements...) { }
// Here, we can't just use 'typename... OtherElements' because MSVC will give
// a warning "C4520: multiple default constructors specified" (even if no one
// actually instantiates the constructor with an empty parameter pack -
// that's probably a bug) and we compile with warnings-as-errors.
template <typename OtherHead, typename... OtherTail,
typename = typename EnableIf<
detail::CheckConvertibility<
detail::Group<OtherHead, OtherTail...>,
detail::Group<Elements...>>::value>::Type>
explicit Tuple(OtherHead&& aHead, OtherTail&&... aTail)
: Impl(Forward<OtherHead>(aHead), Forward<OtherTail>(aTail)...) { }
Tuple(const Tuple& aOther) : Impl(aOther) { }
Tuple(Tuple&& aOther) : Impl(Move(aOther)) { }
Tuple& operator=(const Tuple& aOther)
{
static_cast<Impl&>(*this) = aOther;
return *this;
}
Tuple& operator=(Tuple&& aOther)
{
static_cast<Impl&>(*this) = Move(aOther);
return *this;
}
};
/**
* Specialization of Tuple for zero arguments.
* This is necessary because if the primary template were instantiated with
* an empty parameter pack, the 'Tuple(Elements...)' constructors would
* become illegal overloads of the default constructor.
*/
template <>
class Tuple<> {};
namespace detail {
/*
* Helper functions for implementing Get<N>(tuple).
* These functions take a TupleImpl<Index, Elements...>, with Index being
* explicitly specified, and Elements being deduced. By passing a Tuple
* object as argument, template argument deduction will do its magic and
* cast the tuple to the base class which stores the element at Index.
*/
// Const reference version.
template<std::size_t Index, typename... Elements>
auto TupleGetHelper(TupleImpl<Index, Elements...>& aTuple)
-> decltype(TupleImpl<Index, Elements...>::Head(aTuple))
{
return TupleImpl<Index, Elements...>::Head(aTuple);
}
// Non-const reference version.
template<std::size_t Index, typename... Elements>
auto TupleGetHelper(const TupleImpl<Index, Elements...>& aTuple)
-> decltype(TupleImpl<Index, Elements...>::Head(aTuple))
{
return TupleImpl<Index, Elements...>::Head(aTuple);
}
} // namespace detail
/**
* Index-based access to an element of a tuple.
* The syntax is Get<Index>(tuple). The index is zero-based.
*
* Example:
*
* Tuple<int, float, char> t;
* ...
* float f = Get<1>(t);
*/
// Non-const reference version.
template<std::size_t Index, typename... Elements>
auto Get(Tuple<Elements...>& aTuple)
-> decltype(detail::TupleGetHelper<Index>(aTuple))
{
return detail::TupleGetHelper<Index>(aTuple);
}
// Const reference version.
template<std::size_t Index, typename... Elements>
auto Get(const Tuple<Elements...>& aTuple)
-> decltype(detail::TupleGetHelper<Index>(aTuple))
{
return detail::TupleGetHelper<Index>(aTuple);
}
// Rvalue reference version.
template<std::size_t Index, typename... Elements>
auto Get(Tuple<Elements...>&& aTuple)
-> decltype(Move(mozilla::Get<Index>(aTuple)))
{
// We need a 'mozilla::' qualification here to avoid
// name lookup only finding the current function.
return Move(mozilla::Get<Index>(aTuple));
}
/**
* A convenience function for constructing a tuple out of a sequence of
* values without specifying the type of the tuple.
* The type of the tuple is deduced from the types of its elements.
*
* Example:
*
* auto tuple = MakeTuple(42, 0.5f, 'c'); // has type Tuple<int, float, char>
*/
template<typename... Elements>
Tuple<Elements...> MakeTuple(Elements&&... aElements)
{
return Tuple<Elements...>(Forward<Elements>(aElements)...);
}
} // namespace mozilla
#endif /* mozilla_Tuple_h */
mfbt/moz.build
+1
View File
@@ -81,6 +81,7 @@ EXPORTS.mozilla = [
'TextUtils.h',
'ThreadLocal.h',
'ToString.h',
'Tuple.h',
'TypedEnumBits.h',
'Types.h',
'TypeTraits.h',
mfbt/tests/TestTemplateLib.cpp
+35
View File
@@ -0,0 +1,35 @@
/* -*- Mode: C++; tab-width: 8; indent-tabs-mode: nil; c-basic-offset: 2 -*- */
/* vim: set ts=8 sts=2 et sw=2 tw=80: */
/* This Source Code Form is subject to the terms of the Mozilla Public
* License, v. 2.0. If a copy of the MPL was not distributed with this file,
* You can obtain one at http://mozilla.org/MPL/2.0/. */
#include "mozilla/TemplateLib.h"
using mozilla::tl::And;
static_assert(And<>::value == true,
"And<>::value should be true");
static_assert(And<true>::value == true,
"And<true>::value should be true");
static_assert(And<false>::value == false,
"And<false>::value should be false");
static_assert(And<false, true>::value == false,
"And<false, true>::value should be false");
static_assert(And<false, false>::value == false,
"And<false, false>::value should be false");
static_assert(And<true, false>::value == false,
"And<true, false>::value should be false");
static_assert(And<true, true>::value == true,
"And<true, true>::value should be true");
static_assert(And<true, true, true>::value == true,
"And<true, true, true>::value should be true");
static_assert(And<true, false, true>::value == false,
"And<true, false, true>::value should be false");
int
main()
{
// Nothing to do here.
return 0;
}
mfbt/tests/moz.build
+1
View File
@@ -30,6 +30,7 @@ CppUnitTests([
'TestSegmentedVector',
'TestSHA1',
'TestSplayTree',
'TestTemplateLib',
'TestTextUtils',
'TestTypedEnum',
'TestTypeTraits',
python/mach_commands.py
-1
View File
@@ -9,7 +9,6 @@ import glob
import logging
import mozpack.path as mozpath
import os
import sys
from mozbuild.base import (
MachCommandBase,
python/mozbuild/mozbuild/action/generate_browsersearch.py
-1
View File
@@ -38,7 +38,6 @@ from __future__ import (
import argparse
import codecs
import json
import re
import sys
import os
python/mozbuild/mozbuild/backend/mach_commands.py
-2
View File
@@ -5,8 +5,6 @@
from __future__ import print_function, unicode_literals
import argparse
import glob
import logging
import os
import sys
import subprocess
python/mozbuild/mozbuild/backend/recursivemake.py
-2
View File
@@ -4,8 +4,6 @@
from __future__ import unicode_literals
import errno
import json
import logging
import os
import re
python/mozbuild/mozbuild/base.py
+14
View File
@@ -690,6 +690,20 @@ class MachCommandBase(MozbuildObject):
sys.exit(1)
# Always keep a log of the last command, but don't do that for mach
# invokations from scripts (especially not the ones done by the build
# system itself).
if (self.log_manager and self.log_manager.terminal and
not getattr(self, 'NO_AUTO_LOG', False)):
self._ensure_state_subdir_exists('.')
logfile = self._get_state_filename('last_log.json')
try:
fd = open(logfile, "wb")
self.log_manager.add_json_handler(fd)
except Exception as e:
self.log(logging.WARNING, 'mach', {'error': e},
'Log will not be kept for this command: {error}.')
class MachCommandConditions(object):
"""A series of commonly used condition functions which can be applied to
python/mozbuild/mozbuild/doctor.py
+376
View File
@@ -0,0 +1,376 @@
# This Source Code Form is subject to the terms of the Mozilla Public
# License, v. 2.0. If a copy of the MPL was not distributed with this
# file, # You can obtain one at http://mozilla.org/MPL/2.0/.
import os
import subprocess
import sys
import psutil
from distutils.util import strtobool
from distutils.version import LooseVersion
import mozpack.path as mozpath
# Minimum recommended logical processors in system.
PROCESSORS_THRESHOLD = 4
# Minimum recommended total system memory, in gigabytes.
MEMORY_THRESHOLD = 7.4
# Minimum recommended free space on each disk, in gigabytes.
FREESPACE_THRESHOLD = 10
# Latest MozillaBuild version
LATEST_MOZILLABUILD_VERSION = '1.11.0'
DISABLE_8DOT3_WIN = '''
Disable 8.3 filename creation systemwide?
This increases performance but some legacy applications may not be able to find
files and directories that have long file names.
https://support.microsoft.com/kb/121007
'''
DISABLE_LASTACCESS_WIN = '''
Disable the last access time feature?
This improves the speed of file and
directory access by deferring Last Access Time modification on disk by up to an
hour. Backup programs that rely on this feature may be affected.
https://technet.microsoft.com/en-us/library/cc785435.aspx
'''
class Doctor(object):
def __init__(self, srcdir, objdir, fix):
self.srcdir = mozpath.normpath(srcdir)
self.objdir = mozpath.normpath(objdir)
self.srcdir_mount = self.getmount(self.srcdir)
self.objdir_mount = self.getmount(self.objdir)
self.path_mounts = [
('srcdir', self.srcdir, self.srcdir_mount),
('objdir', self.objdir, self.objdir_mount)
]
self.fix = fix
self.results = []
def check_all(self):
checks = [
'cpu',
'memory',
'storage_freespace',
'fs_8dot3',
'fs_lastaccess',
'mozillabuild'
]
for check in checks:
self.report(getattr(self, check))
good = True
fixable = False
denied = False
for result in self.results:
if result.get('status') != 'GOOD':
good = False
if result.get('fixable', False):
fixable = True
if result.get('denied', False):
denied = True
if denied:
print('run "mach doctor --fix" AS ADMIN to re-attempt fixing your system')
elif False: # elif fixable:
print('run "mach doctor --fix" as admin to attempt fixing your system')
return int(not good)
def getmount(self, path):
while path != '/' and not os.path.ismount(path):
path = mozpath.abspath(mozpath.join(path, os.pardir))
return path
def prompt_bool(self, prompt, limit=5):
''' Prompts the user with prompt and requires a boolean value. '''
valid = False
while not valid and limit > 0:
try:
choice = strtobool(raw_input(prompt + '[Y/N]\n'))
valid = True
except ValueError:
print("ERROR! Please enter a valid option!")
limit -= 1
if limit > 0:
return choice
else:
raise Exception("Error! Reached max attempts of entering option.")
def report(self, results):
# Handle single dict result or list of results.
if isinstance(results, dict):
results = [results]
for result in results:
status = result.get('status', 'UNSURE')
if status == 'SKIPPED':
continue
self.results.append(result)
print('%s...\t%s\n' % (
result.get('desc', ''),
status
)
).expandtabs(40)
@property
def platform(self):
platform = getattr(self, '_platform', None)
if not platform:
platform = sys.platform
while platform[-1].isdigit():
platform = platform[:-1]
setattr(self, '_platform', platform)
return platform
@property
def cpu(self):
cpu_count = psutil.cpu_count()
if cpu_count < PROCESSORS_THRESHOLD:
status = 'BAD'
desc = '%d logical processors detected, <%d' % (
cpu_count, PROCESSORS_THRESHOLD
)
else:
status = 'GOOD'
desc = '%d logical processors detected, >=%d' % (
cpu_count, PROCESSORS_THRESHOLD
)
return {'status': status, 'desc': desc}
@property
def memory(self):
memory = psutil.virtual_memory().total
# Convert to gigabytes.
memory_GB = memory / 1024**3.0
if memory_GB < MEMORY_THRESHOLD:
status = 'BAD'
desc = '%.1fGB of physical memory, <%.1fGB' % (
memory_GB, MEMORY_THRESHOLD
)
else:
status = 'GOOD'
desc = '%.1fGB of physical memory, >%.1fGB' % (
memory_GB, MEMORY_THRESHOLD
)
return {'status': status, 'desc': desc}
@property
def storage_freespace(self):
results = []
desc = ''
mountpoint_line = self.srcdir_mount != self.objdir_mount
for (purpose, path, mount) in self.path_mounts:
desc += '%s = %s\n' % (purpose, path)
if not mountpoint_line:
mountpoint_line = True
continue
try:
usage = psutil.disk_usage(mount)
freespace, size = usage.free, usage.total
freespace_GB = freespace / 1024**3
size_GB = size / 1024**3
if freespace_GB < FREESPACE_THRESHOLD:
status = 'BAD'
desc += 'mountpoint = %s\n%dGB of %dGB free, <%dGB' % (
mount, freespace_GB, size_GB, FREESPACE_THRESHOLD
)
else:
status = 'GOOD'
desc += 'mountpoint = %s\n%dGB of %dGB free, >=%dGB' % (
mount, freespace_GB, size_GB, FREESPACE_THRESHOLD
)
except OSError:
status = 'UNSURE'
desc += 'path invalid'
results.append({'status': status, 'desc': desc})
return results
@property
def fs_8dot3(self):
if self.platform != 'win':
return {'status': 'SKIPPED'}
results = []
fixable = False
denied = False
# See 'fsutil behavior':
# https://technet.microsoft.com/en-us/library/cc785435.aspx
try:
command = 'fsutil behavior query disable8dot3'.split(' ')
fsutil_output = subprocess.check_output(command)
system8dot3 = int(fsutil_output.partition(':')[2][1])
except subprocess.CalledProcessError:
return {'status': 'UNSURE',
'desc': 'unable to check 8dot3 behavior'}
if system8dot3 == 1:
return {'status': 'GOOD',
'desc': '8dot3 disabled systemwide'}
elif system8dot3 == 0:
if False: # if self.fix:
choice = self.prompt_bool(DISABLE_8DOT3_WIN)
if not choice:
return {'status': 'BAD, NOT FIXED',
'desc': '8dot3 enabled systemwide'}
try:
command = 'fsutil behavior set disable8dot3 1'.split(' ')
fsutil_output = subprocess.check_output(command)
status = 'GOOD, FIXED'
desc = '8dot3 disabled systemwide'
except subprocess.CalledProcessError, e:
desc = '8dot3 enabled systemwide'
if e.output.find('denied') != -1:
status = 'BAD, FIX DENIED'
denied = True
else:
status = 'BAD, NOT FIXED'
else:
status = 'BAD, FIXABLE'
desc = '8dot3 enabled systemwide'
fixable = True
return {'status': status, 'desc': desc, 'fixable': fixable,
'denied': denied}
# See 'fsutil 8dot3':
# https://technet.microsoft.com/en-us/library/ff621566.aspx
elif system8dot3 == 2 or system8dot3 == 3:
# 2 = Individual disk behavior respected.
# 3 = 8dot3 disabled on all except system disk.
# Neither is a default value; assume that it's meant to be that
# way and don't try to fix it.
common_mountpoint = self.srcdir_mount == self.objdir_mount
for (purpose, path, mount) in self.path_mounts:
results.append(self.check_disk_8dot3(mount))
if common_mountpoint:
break
return results
def check_disk_8dot3(self, path, disk):
disk = disk.replace('/', '')
try:
command = ('fsutil behavior query disable8dot3 ' + disk).split(' ')
fsutil_output = subprocess.check_output(command)
(volumeLine, systemLine, emptyLine, effectLine, emptyLine2) = fsutil_output.split('\r\n')
volume8dot3 = int(volumeLine.partition(':')[2][1])
effective8dot3 = int(effectLine.find('disabled') != -1)
if volume8dot3 == 1:
# Current disk has 8dot3 disabled.
status = 'GOOD'
desc = '%s has 8dot3 disabled' % disk
else:
status = 'BAD'
desc = '%s has 8dot3 disabled' % disk
except subprocess.CalledProcessError:
status = 'UNSURE'
desc = '%s 8dot3 behavior unknown' % disk
return {'status': status, 'desc': desc}
@property
def fs_lastaccess(self):
results = []
if self.platform == 'win':
fixable = False
denied = False
# See 'fsutil behavior':
# https://technet.microsoft.com/en-us/library/cc785435.aspx
try:
command = 'fsutil behavior query disablelastaccess'.split(' ')
fsutil_output = subprocess.check_output(command)
disablelastaccess = int(fsutil_output.partition('=')[2][1])
except subprocess.CalledProcessError:
disablelastaccess = -1
status = 'UNSURE'
desc = 'unable to check lastaccess behavior'
if disablelastaccess == 1:
status = 'GOOD'
desc = 'lastaccess disabled systemwide'
elif disablelastaccess == 0:
if False: # if self.fix:
choice = self.prompt_bool(DISABLE_LASTACCESS_WIN)
if not choice:
return {'status': 'BAD, NOT FIXED',
'desc': 'lastaccess enabled systemwide'}
try:
command = 'fsutil behavior set disablelastaccess 1'.split(' ')
fsutil_output = subprocess.check_output(command)
status = 'GOOD, FIXED'
desc = 'lastaccess disabled systemwide'
except subprocess.CalledProcessError, e:
desc = 'lastaccess enabled systemwide'
if e.output.find('denied') != -1:
status = 'BAD, FIX DENIED'
denied = True
else:
status = 'BAD, NOT FIXED'
else:
status = 'BAD, FIXABLE'
desc = 'lastaccess enabled'
fixable = True
results.append({'status': status, 'desc': desc, 'fixable': fixable,
'denied': denied})
elif self.platform in ['darwin', 'freebsd', 'linux', 'openbsd']:
common_mountpoint = self.srcdir_mount == self.objdir_mount
for (purpose, path, mount) in self.path_mounts:
results.append(self.check_mount_lastaccess(mount))
if common_mountpoint:
break
else:
results.append({'status': 'SKIPPED'})
return results
def check_mount_lastaccess(self, mount):
partitions = psutil.disk_partitions()
atime_opts = {'atime', 'noatime', 'relatime', 'norelatime'}
option = ''
for partition in partitions:
if partition.mountpoint == mount:
mount_opts = set(partition.opts.split(','))
intersection = list(atime_opts & mount_opts)
if len(intersection) == 1:
option = intersection[0]
break
if not option:
status = 'BAD'
if self.platform == 'linux':
option = 'noatime/relatime'
else:
option = 'noatime'
desc = '%s has no explicit %s mount option' % (
mount, option
)
elif option == 'atime' or option == 'norelatime':
status = 'BAD'
desc = '%s has %s mount option' % (
mount, option
)
elif option == 'noatime' or option == 'relatime':
status = 'GOOD'
desc = '%s has %s mount option' % (
mount, option
)
return {'status': status, 'desc': desc}
@property
def mozillabuild(self):
if self.platform != 'win':
return {'status': 'SKIPPED'}
MOZILLABUILD = mozpath.normpath(os.environ.get('MOZILLABUILD', ''))
if not MOZILLABUILD or not os.path.exists(MOZILLABUILD):
return {'desc': 'not running under MozillaBuild'}
try:
with open(mozpath.join(MOZILLABUILD, 'VERSION'), 'r') as fh:
version = fh.readline()
if not version:
raise ValueError()
if LooseVersion(version) < LooseVersion(LATEST_MOZILLABUILD_VERSION):
status = 'BAD'
desc = 'MozillaBuild %s in use, <%s' % (
version, LATEST_MOZILLABUILD_VERSION
)
else:
status = 'GOOD'
desc = 'MozillaBuild %s in use' % version
except (IOError, ValueError):
status = 'UNSURE'
desc = 'MozillaBuild version not found'
return {'status': status, 'desc': desc}
python/mozbuild/mozbuild/dotproperties.py
-1
View File
@@ -10,7 +10,6 @@ from __future__ import unicode_literals
import codecs
import re
import os
import sys
if sys.version_info[0] == 3:
python/mozbuild/mozbuild/frontend/context.py
+125 -47
View File
@@ -333,45 +333,110 @@ def Enum(*values):
return EnumClass
class SourcePath(ContextDerivedValue, UserString):
class PathMeta(type):
"""Meta class for the Path family of classes.
It handles calling __new__ and __init__ with the right arguments
in cases where a Path is instantiated with another instance of
Path instead of having received a context.
It also makes Path(context, value) instantiate one of the
subclasses depending on the value, allowing callers to do
standard type checking (isinstance(path, ObjDirPath)) instead
of checking the value itself (path.startswith('!')).
"""
def __call__(cls, context, value=None):
if isinstance(context, Path):
assert value is None
value = context
context = context.context
else:
assert isinstance(context, Context)
if isinstance(value, Path):
context = value.context
if not issubclass(cls, (SourcePath, ObjDirPath)):
cls = ObjDirPath if value.startswith('!') else SourcePath
return super(PathMeta, cls).__call__(context, value)
class Path(ContextDerivedValue, unicode):
"""Stores and resolves a source path relative to a given context
This class is used as a backing type for some of the sandbox variables.
It expresses paths relative to a context. Paths starting with a '/'
are considered relative to the topsrcdir, and other paths relative
to the current source directory for the associated context.
It expresses paths relative to a context. Supported paths are:
- '/topsrcdir/relative/paths'
- 'srcdir/relative/paths'
- '!/topobjdir/relative/paths'
- '!objdir/relative/paths'
"""
__metaclass__ = PathMeta
def __new__(cls, context, value=None):
if not isinstance(context, Context) and value is None:
return unicode(context)
return super(SourcePath, cls).__new__(cls)
return super(Path, cls).__new__(cls, value)
def __init__(self, context, value=None):
# Only subclasses should be instantiated.
assert self.__class__ != Path
self.context = context
self.srcdir = context.srcdir
self.value = value
@memoized_property
def data(self):
"""Serializes the path for UserString."""
if self.value.startswith('/'):
ret = None
def join(self, *p):
"""ContextDerived equivalent of mozpath.join(self, *p), returning a
new Path instance.
"""
return Path(self.context, mozpath.join(self, *p))
def __cmp__(self, other):
if isinstance(other, Path) and self.srcdir != other.srcdir:
return cmp(self.full_path, other.full_path)
return cmp(unicode(self), other)
# __cmp__ is not enough because unicode has __eq__, __ne__, etc. defined
# and __cmp__ is only used for those when they don't exist.
def __eq__(self, other):
return self.__cmp__(other) == 0
def __ne__(self, other):
return self.__cmp__(other) != 0
def __lt__(self, other):
return self.__cmp__(other) < 0
def __gt__(self, other):
return self.__cmp__(other) > 0
def __le__(self, other):
return self.__cmp__(other) <= 0
def __ge__(self, other):
return self.__cmp__(other) >= 0
def __repr__(self):
return '<%s (%s)%s>' % (self.__class__.__name__, self.srcdir, self)
def __hash__(self):
return hash(self.full_path)
class SourcePath(Path):
"""Like Path, but limited to paths in the source directory."""
def __init__(self, context, value):
if value.startswith('!'):
raise ValueError('Object directory paths are not allowed')
super(SourcePath, self).__init__(context, value)
if value.startswith('/'):
path = None
# If the path starts with a '/' and is actually relative to an
# external source dir, use that as base instead of topsrcdir.
if self.context.config.external_source_dir:
ret = mozpath.join(self.context.config.external_source_dir,
self.value[1:])
if not ret or not os.path.exists(ret):
ret = mozpath.join(self.context.config.topsrcdir,
self.value[1:])
if context.config.external_source_dir:
path = mozpath.join(context.config.external_source_dir,
value[1:])
if not path or not os.path.exists(path):
path = mozpath.join(context.config.topsrcdir,
value[1:])
else:
ret = mozpath.join(self.srcdir, self.value)
return mozpath.normpath(ret)
def __unicode__(self):
# UserString doesn't implement a __unicode__ function at all, so add
# ours.
return self.data
path = mozpath.join(self.srcdir, value)
self.full_path = mozpath.normpath(path)
@memoized_property
def translated(self):
@@ -381,38 +446,51 @@ class SourcePath(ContextDerivedValue, UserString):
path under topsrcdir and the external source dir end up mixed in the
objdir (aka pseudo-rework), this is needed.
"""
if self.value.startswith('/'):
ret = mozpath.join(self.context.config.topobjdir, self.value[1:])
return ObjDirPath(self.context, '!%s' % self).full_path
class ObjDirPath(Path):
"""Like Path, but limited to paths in the object directory."""
def __init__(self, context, value=None):
if not value.startswith('!'):
raise ValueError('Source paths are not allowed')
super(ObjDirPath, self).__init__(context, value)
if value.startswith('!/'):
path = mozpath.join(context.config.topobjdir,value[2:])
else:
ret = mozpath.join(self.context.objdir, self.value)
return mozpath.normpath(ret)
def join(self, *p):
"""Lazy mozpath.join(self, *p), returning a new SourcePath instance.
In an ideal world, this wouldn't be required, but with the
external_source_dir business, and the fact that comm-central and
mozilla-central have directories in common, resolving a SourcePath
before doing mozpath.join doesn't work out properly.
"""
return SourcePath(self.context, mozpath.join(self.value, *p))
path = mozpath.join(context.objdir, value[1:])
self.full_path = mozpath.normpath(path)
@memoize
def ContextDerivedTypedList(type, base_class=List):
def ContextDerivedTypedList(klass, base_class=List):
"""Specialized TypedList for use with ContextDerivedValue types.
"""
assert issubclass(type, ContextDerivedValue)
class _TypedList(ContextDerivedValue, TypedList(type, base_class)):
assert issubclass(klass, ContextDerivedValue)
class _TypedList(ContextDerivedValue, TypedList(klass, base_class)):
def __init__(self, context, iterable=[]):
class _Type(type):
def __new__(cls, obj):
return type(context, obj)
self.TYPE = _Type
self.context = context
super(_TypedList, self).__init__(iterable)
def normalize(self, e):
if not isinstance(e, klass):
e = klass(self.context, e)
return e
return _TypedList
@memoize
def ContextDerivedTypedListWithItems(type, base_class=List):
"""Specialized TypedList for use with ContextDerivedValue types.
"""
class _TypedListWithItems(ContextDerivedTypedList(type, base_class)):
def __getitem__(self, name):
name = self.normalize(name)
return super(_TypedListWithItems, self).__getitem__(name)
return _TypedListWithItems
BugzillaComponent = TypedNamedTuple('BugzillaComponent',
[('product', unicode), ('component', unicode)])
python/mozbuild/mozbuild/frontend/data.py
-3
View File
@@ -17,9 +17,6 @@ structures.
from __future__ import unicode_literals
import os
from collections import OrderedDict
from mozbuild.util import (
shell_quote,
StrictOrderingOnAppendList,
python/mozbuild/mozbuild/frontend/emitter.py
-1
View File
@@ -5,7 +5,6 @@
from __future__ import unicode_literals
import itertools
import json
import logging
import os
import traceback
python/mozbuild/mozbuild/frontend/reader.py
+6 -5
View File
@@ -297,8 +297,8 @@ class MozbuildSandbox(Sandbox):
def _include(self, path):
"""Include and exec another file within the context of this one."""
# path is a SourcePath, and needs to be coerced to unicode.
self.exec_file(unicode(path))
# path is a SourcePath
self.exec_file(path.full_path)
def _warning(self, message):
# FUTURE consider capturing warnings in a variable instead of printing.
@@ -432,7 +432,7 @@ class MozbuildSandbox(Sandbox):
for k, v in inspect.getcallargs(func, *args, **kwargs).items():
sandbox[k] = v
sandbox.exec_source(code, path)
sandbox.exec_source(code, path, becomes_current_path=False)
# This is gross, but allows the merge to happen. Eventually, the
# merging will go away and template contexts emitted independently.
@@ -1100,7 +1100,8 @@ class BuildReader(object):
if d in recurse_info:
raise SandboxValidationError(
'Directory (%s) registered multiple times in %s' % (
mozpath.relpath(d, context.srcdir), var), context)
mozpath.relpath(d.full_path, context.srcdir), var),
context)
recurse_info[d] = {}
for key in sandbox.metadata:
@@ -1110,7 +1111,7 @@ class BuildReader(object):
recurse_info[d][key] = dict(sandbox.metadata[key])
for path, child_metadata in recurse_info.items():
child_path = path.join('moz.build')
child_path = path.join('moz.build').full_path
# Ensure we don't break out of the topsrcdir. We don't do realpath
# because it isn't necessary. If there are symlinks in the srcdir,
python/mozbuild/mozbuild/frontend/sandbox.py
+15 -7
View File
@@ -151,7 +151,7 @@ class Sandbox(dict):
self.exec_source(source, path)
def exec_source(self, source, path=''):
def exec_source(self, source, path='', becomes_current_path=True):
"""Execute Python code within a string.
The passed string should contain Python code to be executed. The string
@@ -161,7 +161,7 @@ class Sandbox(dict):
does not perform extra path normalization. This can cause relative
paths to behave weirdly.
"""
if path:
if path and becomes_current_path:
self._context.push_source(path)
old_sandbox = self._context._sandbox
@@ -193,16 +193,24 @@ class Sandbox(dict):
if self._last_name_error is not None:
actual = self._last_name_error
raise SandboxExecutionError(self._context.source_stack,
type(actual), actual, sys.exc_info()[2])
source_stack = self._context.source_stack
if not becomes_current_path:
# Add current file to the stack because it wasn't added before
# sandbox execution.
source_stack.append(path)
raise SandboxExecutionError(source_stack, type(actual), actual,
sys.exc_info()[2])
except Exception as e:
# Need to copy the stack otherwise we get a reference and that is
# mutated during the finally.
exc = sys.exc_info()
raise SandboxExecutionError(self._context.source_stack, exc[0],
exc[1], exc[2])
source_stack = self._context.source_stack
if not becomes_current_path:
# Add current file to the stack because it wasn't added before
# sandbox execution.
source_stack.append(path)
raise SandboxExecutionError(source_stack, exc[0], exc[1], exc[2])
finally:
self._context._sandbox = old_sandbox
if path:
python/mozbuild/mozbuild/mach_commands.py
+74 -1
View File
@@ -4,10 +4,12 @@
from __future__ import print_function, unicode_literals
import itertools
import argparse
import json
import logging
import operator
import os
import subprocess
import sys
import mozpack.path as mozpath
@@ -553,6 +555,77 @@ class Build(MachCommandBase):
return self._run_command_in_objdir(args=args, pass_thru=True,
ensure_exit_code=False)
@CommandProvider
class Doctor(MachCommandBase):
"""Provide commands for diagnosing common build environment problems"""
@Command('doctor', category='devenv',
description='')
@CommandArgument('--fix', default=None, action='store_true',
help='Attempt to fix found problems.')
def doctor(self, fix=None):
self._activate_virtualenv()
from mozbuild.doctor import Doctor
doctor = Doctor(self.topsrcdir, self.topobjdir, fix)
return doctor.check_all()
@CommandProvider
class Logs(MachCommandBase):
"""Provide commands to read mach logs."""
NO_AUTO_LOG = True
@Command('show-log', category='post-build',
description='Display mach logs')
@CommandArgument('log_file', nargs='?', type=argparse.FileType('rb'),
help='Filename to read log data from. Defaults to the log of the last '
'mach command.')
def show_log(self, log_file=None):
if not log_file:
path = self._get_state_filename('last_log.json')
log_file = open(path, 'rb')
if self.log_manager.terminal:
env = dict(os.environ)
if 'LESS' not in env:
# Sensible default flags if none have been set in the user
# environment.
env['LESS'] = 'FRX'
less = subprocess.Popen(['less'], stdin=subprocess.PIPE, env=env)
# Various objects already have a reference to sys.stdout, so we
# can't just change it, we need to change the file descriptor under
# it to redirect to less's input.
# First keep a copy of the sys.stdout file descriptor.
output_fd = os.dup(sys.stdout.fileno())
os.dup2(less.stdin.fileno(), sys.stdout.fileno())
startTime = 0
for line in log_file:
created, action, params = json.loads(line)
if not startTime:
startTime = created
self.log_manager.terminal_handler.formatter.start_time = \
created
if 'line' in params:
record = logging.makeLogRecord({
'created': created,
'name': self._logger.name,
'levelno': logging.INFO,
'msg': '{line}',
'params': params,
'action': action,
})
self._logger.handle(record)
if self.log_manager.terminal:
# Close less's input so that it knows that we're done sending data.
less.stdin.close()
# Since the less's input file descriptor is now also the stdout
# file descriptor, we still actually have a non-closed system file
# descriptor for less's input. Replacing sys.stdout's file
# descriptor with what it was before we replaced it will properly
# close less's input.
os.dup2(output_fd, sys.stdout.fileno())
less.wait()
@CommandProvider
class Warnings(MachCommandBase):
python/mozbuild/mozbuild/preprocessor.py
-1
View File
@@ -24,7 +24,6 @@ value :
import sys
import os
import os.path
import re
from optparse import OptionParser
import errno
python/mozbuild/mozbuild/test/frontend/test_context.py
+303 -2
View File
@@ -9,12 +9,18 @@ from mozunit import main
from mozbuild.frontend.context import (
Context,
ContextDerivedTypedList,
ContextDerivedTypedListWithItems,
FUNCTIONS,
ObjDirPath,
Path,
SourcePath,
SPECIAL_VARIABLES,
SUBCONTEXTS,
VARIABLES,
)
from mozbuild.util import StrictOrderingOnAppendListWithFlagsFactory
from mozpack import path as mozpath
@@ -99,7 +105,7 @@ class TestContext(unittest.TestCase):
self.assertEqual(test['foo'], 42)
self.assertEqual(test['baz'], { 'c': 3, 'd': 4 })
def test_paths(self):
def test_context_paths(self):
test = Context()
# Newly created context has no paths.
@@ -203,7 +209,7 @@ class TestContext(unittest.TestCase):
self.assertEqual(test.all_paths, set([bar, foo]))
self.assertEqual(test.source_stack, [foo, bar, bar, foo])
def test_dirs(self):
def test_context_dirs(self):
class Config(object): pass
config = Config()
config.topsrcdir = mozpath.abspath(os.curdir)
@@ -264,5 +270,300 @@ class TestSymbols(unittest.TestCase):
self._verify_doc(v[2])
class TestPaths(unittest.TestCase):
@classmethod
def setUpClass(cls):
class Config(object): pass
cls.config = config = Config()
config.topsrcdir = mozpath.abspath(os.curdir)
config.topobjdir = mozpath.abspath('obj')
config.external_source_dir = None
def test_path(self):
config = self.config
ctxt1 = Context(config=config)
ctxt1.push_source(mozpath.join(config.topsrcdir, 'foo', 'moz.build'))
ctxt2 = Context(config=config)
ctxt2.push_source(mozpath.join(config.topsrcdir, 'bar', 'moz.build'))
path1 = Path(ctxt1, 'qux')
self.assertIsInstance(path1, SourcePath)
self.assertEqual(path1, 'qux')
self.assertEqual(path1.full_path,
mozpath.join(config.topsrcdir, 'foo', 'qux'))
path2 = Path(ctxt2, '../foo/qux')
self.assertIsInstance(path2, SourcePath)
self.assertEqual(path2, '../foo/qux')
self.assertEqual(path2.full_path,
mozpath.join(config.topsrcdir, 'foo', 'qux'))
self.assertEqual(path1, path2)
self.assertEqual(path1.join('../../bar/qux').full_path,
mozpath.join(config.topsrcdir, 'bar', 'qux'))
path1 = Path(ctxt1, '/qux/qux')
self.assertIsInstance(path1, SourcePath)
self.assertEqual(path1, '/qux/qux')
self.assertEqual(path1.full_path,
mozpath.join(config.topsrcdir, 'qux', 'qux'))
path2 = Path(ctxt2, '/qux/qux')
self.assertIsInstance(path2, SourcePath)
self.assertEqual(path2, '/qux/qux')
self.assertEqual(path2.full_path,
mozpath.join(config.topsrcdir, 'qux', 'qux'))
self.assertEqual(path1, path2)
path1 = Path(ctxt1, '!qux')
self.assertIsInstance(path1, ObjDirPath)
self.assertEqual(path1, '!qux')
self.assertEqual(path1.full_path,
mozpath.join(config.topobjdir, 'foo', 'qux'))
path2 = Path(ctxt2, '!../foo/qux')
self.assertIsInstance(path2, ObjDirPath)
self.assertEqual(path2, '!../foo/qux')
self.assertEqual(path2.full_path,
mozpath.join(config.topobjdir, 'foo', 'qux'))
self.assertEqual(path1, path2)
path1 = Path(ctxt1, '!/qux/qux')
self.assertIsInstance(path1, ObjDirPath)
self.assertEqual(path1, '!/qux/qux')
self.assertEqual(path1.full_path,
mozpath.join(config.topobjdir, 'qux', 'qux'))
path2 = Path(ctxt2, '!/qux/qux')
self.assertIsInstance(path2, ObjDirPath)
self.assertEqual(path2, '!/qux/qux')
self.assertEqual(path2.full_path,
mozpath.join(config.topobjdir, 'qux', 'qux'))
self.assertEqual(path1, path2)
path1 = Path(ctxt1, path1)
self.assertIsInstance(path1, ObjDirPath)
self.assertEqual(path1, '!/qux/qux')
self.assertEqual(path1.full_path,
mozpath.join(config.topobjdir, 'qux', 'qux'))
path2 = Path(ctxt2, path2)
self.assertIsInstance(path2, ObjDirPath)
self.assertEqual(path2, '!/qux/qux')
self.assertEqual(path2.full_path,
mozpath.join(config.topobjdir, 'qux', 'qux'))
self.assertEqual(path1, path2)
path1 = Path(path1)
self.assertIsInstance(path1, ObjDirPath)
self.assertEqual(path1, '!/qux/qux')
self.assertEqual(path1.full_path,
mozpath.join(config.topobjdir, 'qux', 'qux'))
self.assertEqual(path1, path2)
path2 = Path(path2)
self.assertIsInstance(path2, ObjDirPath)
self.assertEqual(path2, '!/qux/qux')
self.assertEqual(path2.full_path,
mozpath.join(config.topobjdir, 'qux', 'qux'))
self.assertEqual(path1, path2)
def test_source_path(self):
config = self.config
ctxt = Context(config=config)
ctxt.push_source(mozpath.join(config.topsrcdir, 'foo', 'moz.build'))
path = SourcePath(ctxt, 'qux')
self.assertEqual(path, 'qux')
self.assertEqual(path.full_path,
mozpath.join(config.topsrcdir, 'foo', 'qux'))
self.assertEqual(path.translated,
mozpath.join(config.topobjdir, 'foo', 'qux'))
path = SourcePath(ctxt, '../bar/qux')
self.assertEqual(path, '../bar/qux')
self.assertEqual(path.full_path,
mozpath.join(config.topsrcdir, 'bar', 'qux'))
self.assertEqual(path.translated,
mozpath.join(config.topobjdir, 'bar', 'qux'))
path = SourcePath(ctxt, '/qux/qux')
self.assertEqual(path, '/qux/qux')
self.assertEqual(path.full_path,
mozpath.join(config.topsrcdir, 'qux', 'qux'))
self.assertEqual(path.translated,
mozpath.join(config.topobjdir, 'qux', 'qux'))
with self.assertRaises(ValueError):
SourcePath(ctxt, '!../bar/qux')
with self.assertRaises(ValueError):
SourcePath(ctxt, '!/qux/qux')
path = SourcePath(path)
self.assertIsInstance(path, SourcePath)
self.assertEqual(path, '/qux/qux')
self.assertEqual(path.full_path,
mozpath.join(config.topsrcdir, 'qux', 'qux'))
self.assertEqual(path.translated,
mozpath.join(config.topobjdir, 'qux', 'qux'))
path = Path(path)
self.assertIsInstance(path, SourcePath)
def test_objdir_path(self):
config = self.config
ctxt = Context(config=config)
ctxt.push_source(mozpath.join(config.topsrcdir, 'foo', 'moz.build'))
path = ObjDirPath(ctxt, '!qux')
self.assertEqual(path, '!qux')
self.assertEqual(path.full_path,
mozpath.join(config.topobjdir, 'foo', 'qux'))
path = ObjDirPath(ctxt, '!../bar/qux')
self.assertEqual(path, '!../bar/qux')
self.assertEqual(path.full_path,
mozpath.join(config.topobjdir, 'bar', 'qux'))
path = ObjDirPath(ctxt, '!/qux/qux')
self.assertEqual(path, '!/qux/qux')
self.assertEqual(path.full_path,
mozpath.join(config.topobjdir, 'qux', 'qux'))
with self.assertRaises(ValueError):
path = ObjDirPath(ctxt, '../bar/qux')
with self.assertRaises(ValueError):
path = ObjDirPath(ctxt, '/qux/qux')
path = ObjDirPath(path)
self.assertIsInstance(path, ObjDirPath)
self.assertEqual(path, '!/qux/qux')
self.assertEqual(path.full_path,
mozpath.join(config.topobjdir, 'qux', 'qux'))
path = Path(path)
self.assertIsInstance(path, ObjDirPath)
def test_path_with_mixed_contexts(self):
config = self.config
ctxt1 = Context(config=config)
ctxt1.push_source(mozpath.join(config.topsrcdir, 'foo', 'moz.build'))
ctxt2 = Context(config=config)
ctxt2.push_source(mozpath.join(config.topsrcdir, 'bar', 'moz.build'))
path1 = Path(ctxt1, 'qux')
path2 = Path(ctxt2, path1)
self.assertEqual(path2, path1)
self.assertEqual(path2, 'qux')
self.assertEqual(path2.context, ctxt1)
self.assertEqual(path2.full_path,
mozpath.join(config.topsrcdir, 'foo', 'qux'))
path1 = Path(ctxt1, '../bar/qux')
path2 = Path(ctxt2, path1)
self.assertEqual(path2, path1)
self.assertEqual(path2, '../bar/qux')
self.assertEqual(path2.context, ctxt1)
self.assertEqual(path2.full_path,
mozpath.join(config.topsrcdir, 'bar', 'qux'))
path1 = Path(ctxt1, '/qux/qux')
path2 = Path(ctxt2, path1)
self.assertEqual(path2, path1)
self.assertEqual(path2, '/qux/qux')
self.assertEqual(path2.context, ctxt1)
self.assertEqual(path2.full_path,
mozpath.join(config.topsrcdir, 'qux', 'qux'))
path1 = Path(ctxt1, '!qux')
path2 = Path(ctxt2, path1)
self.assertEqual(path2, path1)
self.assertEqual(path2, '!qux')
self.assertEqual(path2.context, ctxt1)
self.assertEqual(path2.full_path,
mozpath.join(config.topobjdir, 'foo', 'qux'))
path1 = Path(ctxt1, '!../bar/qux')
path2 = Path(ctxt2, path1)
self.assertEqual(path2, path1)
self.assertEqual(path2, '!../bar/qux')
self.assertEqual(path2.context, ctxt1)
self.assertEqual(path2.full_path,
mozpath.join(config.topobjdir, 'bar', 'qux'))
path1 = Path(ctxt1, '!/qux/qux')
path2 = Path(ctxt2, path1)
self.assertEqual(path2, path1)
self.assertEqual(path2, '!/qux/qux')
self.assertEqual(path2.context, ctxt1)
self.assertEqual(path2.full_path,
mozpath.join(config.topobjdir, 'qux', 'qux'))
def test_path_typed_list(self):
config = self.config
ctxt1 = Context(config=config)
ctxt1.push_source(mozpath.join(config.topsrcdir, 'foo', 'moz.build'))
ctxt2 = Context(config=config)
ctxt2.push_source(mozpath.join(config.topsrcdir, 'bar', 'moz.build'))
paths = [
'!../bar/qux',
'!/qux/qux',
'!qux',
'../bar/qux',
'/qux/qux',
'qux',
]
MyList = ContextDerivedTypedList(Path)
l = MyList(ctxt1)
l += paths
for p_str, p_path in zip(paths, l):
self.assertEqual(p_str, p_path)
self.assertEqual(p_path, Path(ctxt1, p_str))
self.assertEqual(p_path.join('foo'),
Path(ctxt1, mozpath.join(p_str, 'foo')))
l2 = MyList(ctxt2)
l2 += paths
for p_str, p_path in zip(paths, l2):
self.assertEqual(p_str, p_path)
self.assertEqual(p_path, Path(ctxt2, p_str))
# Assigning with Paths from another context doesn't rebase them
l2 = MyList(ctxt2)
l2 += l
for p_str, p_path in zip(paths, l2):
self.assertEqual(p_str, p_path)
self.assertEqual(p_path, Path(ctxt1, p_str))
MyListWithFlags = ContextDerivedTypedListWithItems(
Path, StrictOrderingOnAppendListWithFlagsFactory({
'foo': bool,
}))
l = MyListWithFlags(ctxt1)
l += paths
for p in paths:
l[p].foo = True
for p_str, p_path in zip(paths, l):
self.assertEqual(p_str, p_path)
self.assertEqual(p_path, Path(ctxt1, p_str))
self.assertEqual(l[p_str].foo, True)
self.assertEqual(l[p_path].foo, True)
if __name__ == '__main__':
main()
python/mozbuild/mozbuild/test/frontend/test_emitter.py
+3 -3
View File
@@ -106,7 +106,7 @@ class TestEmitterBasic(unittest.TestCase):
self.assertEqual(objs[3].affected_tiers, {'misc'})
dirs = [o.dirs for o in objs]
dirs = [[d.full_path for d in o.dirs] for o in objs]
self.assertEqual(dirs, [
[
mozpath.join(reader.config.topsrcdir, 'foo'),
@@ -130,9 +130,9 @@ class TestEmitterBasic(unittest.TestCase):
reldir = o.relativedir
if reldir == '':
self.assertEqual(o.dirs, [
self.assertEqual([d.full_path for d in o.dirs], [
mozpath.join(reader.config.topsrcdir, 'regular')])
self.assertEqual(o.test_dirs, [
self.assertEqual([d.full_path for d in o.test_dirs], [
mozpath.join(reader.config.topsrcdir, 'test')])
def test_config_file_substitution(self):
python/mozbuild/mozbuild/test/frontend/test_namespaces.py
+37 -2
View File
@@ -12,10 +12,12 @@ from mozbuild.frontend.context import (
Context,
ContextDerivedValue,
ContextDerivedTypedList,
ContextDerivedTypedListWithItems,
)
from mozbuild.util import (
StrictOrderingOnAppendList,
StrictOrderingOnAppendListWithFlagsFactory,
UnsortedError,
)
@@ -38,6 +40,12 @@ class Piyo(ContextDerivedValue):
def __str__(self):
return self.value
def __cmp__(self, other):
return cmp(self.value, str(other))
def __hash__(self):
return hash(self.value)
VARIABLES = {
'HOGE': (unicode, unicode, None, None),
@@ -45,6 +53,11 @@ VARIABLES = {
'PIYO': (Piyo, unicode, None, None),
'HOGERA': (ContextDerivedTypedList(Piyo, StrictOrderingOnAppendList),
list, None, None),
'HOGEHOGE': (ContextDerivedTypedListWithItems(
Piyo,
StrictOrderingOnAppendListWithFlagsFactory({
'foo': bool,
})), list, None, None),
}
class TestContext(unittest.TestCase):
@@ -158,8 +171,7 @@ class TestContext(unittest.TestCase):
ns['HOGERA'] += ['a', 'b', 'c']
self.assertIsInstance(ns['HOGERA'],
ContextDerivedTypedList(Piyo, StrictOrderingOnAppendList))
self.assertIsInstance(ns['HOGERA'], VARIABLES['HOGERA'][0])
for n in range(0, 3):
self.assertIsInstance(ns['HOGERA'][n], Piyo)
self.assertEqual(ns['HOGERA'][n].value, ['a', 'b', 'c'][n])
@@ -168,5 +180,28 @@ class TestContext(unittest.TestCase):
with self.assertRaises(UnsortedError):
ns['HOGERA'] += ['f', 'e', 'd']
def test_context_derived_typed_list_with_items(self):
ns = Context(allowed_variables=VARIABLES)
# Setting to a type that's rejected by coercion should not work.
with self.assertRaises(ValueError):
ns['HOGEHOGE'] = [False]
values = ['a', 'b', 'c']
ns['HOGEHOGE'] += values
self.assertIsInstance(ns['HOGEHOGE'], VARIABLES['HOGEHOGE'][0])
for v in values:
ns['HOGEHOGE'][v].foo = True
for v, item in zip(values, ns['HOGEHOGE']):
self.assertIsInstance(item, Piyo)
self.assertEqual(v, item)
self.assertEqual(ns['HOGEHOGE'][v].foo, True)
self.assertEqual(ns['HOGEHOGE'][item].foo, True)
with self.assertRaises(UnsortedError):
ns['HOGEHOGE'] += ['f', 'e', 'd']
if __name__ == '__main__':
main()
python/mozbuild/mozbuild/test/frontend/test_sandbox.py
+11 -7
View File
@@ -24,6 +24,7 @@ from mozbuild.frontend.sandbox import (
from mozbuild.frontend.context import (
Context,
FUNCTIONS,
SourcePath,
SPECIAL_VARIABLES,
VARIABLES,
)
@@ -132,6 +133,9 @@ class TestedSandbox(MozbuildSandbox):
return mozpath.normpath(
mozpath.join(self._context.config.topsrcdir, path))
def source_path(self, path):
return SourcePath(self._context, path)
def exec_file(self, path):
super(TestedSandbox, self).exec_file(self.normalize_path(path))
@@ -250,8 +254,8 @@ class TestMozbuildSandbox(unittest.TestCase):
sandbox.exec_file('moz.build')
self.assertEqual(sandbox['DIRS'], [
sandbox.normalize_path('foo'),
sandbox.normalize_path('bar'),
sandbox.source_path('foo'),
sandbox.source_path('bar'),
])
self.assertEqual(sandbox._context.main_path,
sandbox.normalize_path('moz.build'))
@@ -299,11 +303,11 @@ class TestMozbuildSandbox(unittest.TestCase):
# child directory.
sandbox = self.sandbox(data_path='include-relative-from-child')
sandbox.exec_file('child/child.build')
self.assertEqual(sandbox['DIRS'], [sandbox.normalize_path('foo')])
self.assertEqual(sandbox['DIRS'], [sandbox.source_path('../foo')])
sandbox = self.sandbox(data_path='include-relative-from-child')
sandbox.exec_file('child/child2.build')
self.assertEqual(sandbox['DIRS'], [sandbox.normalize_path('foo')])
self.assertEqual(sandbox['DIRS'], [sandbox.source_path('../foo')])
def test_include_topsrcdir_relative(self):
# An absolute path for include() is relative to topsrcdir.
@@ -311,7 +315,7 @@ class TestMozbuildSandbox(unittest.TestCase):
sandbox = self.sandbox(data_path='include-topsrcdir-relative')
sandbox.exec_file('moz.build')
self.assertEqual(sandbox['DIRS'], [sandbox.normalize_path('foo')])
self.assertEqual(sandbox['DIRS'], [sandbox.source_path('foo')])
def test_error(self):
sandbox = self.sandbox()
@@ -382,7 +386,7 @@ SOURCES += ['hoge.cpp']
self.assertEqual(sandbox2._context, {
'SOURCES': ['qux.cpp', 'bar.cpp', 'foo.cpp', 'hoge.cpp'],
'DIRS': [sandbox.normalize_path('foo')],
'DIRS': [sandbox2.source_path('foo')],
})
sandbox2 = self.sandbox(metadata={'templates': sandbox.templates})
@@ -425,7 +429,7 @@ TemplateGlobalUPPERVariable()
sandbox2.exec_source(source, 'foo.mozbuild')
self.assertEqual(sandbox2._context, {
'SOURCES': [],
'DIRS': [sandbox2.normalize_path('foo')],
'DIRS': [sandbox2.source_path('foo')],
})
# However, the result of the template is mixed with the global
python/mozbuild/mozbuild/util.py
+7 -8
View File
@@ -8,7 +8,6 @@
from __future__ import unicode_literals
import collections
import copy
import difflib
import errno
import functools
@@ -886,12 +885,7 @@ class TypedListMixin(object):
if isinstance(l, self.__class__):
return l
def normalize(e):
if not isinstance(e, self.TYPE):
e = self.TYPE(e)
return e
return [normalize(e) for e in l]
return [self.normalize(e) for e in l]
def __init__(self, iterable=[]):
iterable = self._ensure_type(iterable)
@@ -936,7 +930,12 @@ def TypedList(type, base_class=List):
TypedList(unicode, StrictOrderingOnAppendList)
'''
class _TypedList(TypedListMixin, base_class):
TYPE = type
@staticmethod
def normalize(e):
if not isinstance(e, type):
e = type(e)
return e
return _TypedList
def group_unified_files(files, unified_prefix, unified_suffix,
services/common/tests/mach_commands.py
-5
View File
@@ -4,12 +4,7 @@
from __future__ import unicode_literals
import logging
import mozpack.path as mozpath
import os
import sys
import warnings
import which
from mozbuild.base import (
MachCommandBase,
testing/cppunittest.ini
+1
View File
@@ -87,6 +87,7 @@ skip-if = os == 'b2g' #Bug 919595
[TestWebGLElementArrayCache]
[buffered_stun_socket_unittest]
[ice_unittest]
[test_nr_socket_unittest]
[jsapi-tests]
skip-if = os == 'b2g' #Bug 1068946
[mediaconduit_unittests]
testing/docker/b2g-build/Dockerfile
+1
View File
@@ -11,6 +11,7 @@ RUN yum install -y epel-release && \
GConf2-devel \
alsa-lib-devel \
autoconf213 \
bc \
bison \
bzip2 \
ccache \
testing/docker/b2g-build/VERSION
+1 -1
View File
@@ -1 +1 @@
0.2.6
0.2.7
testing/docker/builder/Dockerfile
+2 -2
View File
@@ -1,4 +1,4 @@
FROM quay.io/mozilla/b2g-build:0.2.6
FROM quay.io/mozilla/b2g-build:0.2.7
MAINTAINER Dustin J. Mitchell <dustin@mozilla.com>
ENV PYTHONPATH /tools/tools/lib/python:$PYTHONPATH
@@ -21,7 +21,7 @@ RUN git config --global user.email "mozilla@example.com" && \
git config --global user.name "mozilla"
# VCS Tools
RUN npm install -g taskcluster-vcs@2.3.1
RUN npm install -g taskcluster-vcs@2.3.5
# TODO enable worker
# TODO volume mount permissions will be an issue
testing/docker/builder/REGISTRY
+1
View File
@@ -0,0 +1 @@
taskcluster
testing/docker/builder/VERSION
+1 -1
View File
@@ -1 +1 @@
0.5.2
0.5.4
testing/docker/decision/Dockerfile
+1 -1
View File
@@ -4,5 +4,5 @@ MAINTAINER Jonas Finnemann Jensen <jopsen@gmail.com>
ENV PATH /home/worker/bin/:$PATH
# Add utilities and configuration
RUN npm install -g taskcluster-vcs@2.3.1
RUN npm install -g taskcluster-vcs@2.3.4
ADD bin /home/worker/bin
testing/docker/decision/VERSION
+1 -1
View File
@@ -1 +1 @@
0.0.4
0.0.5
testing/docker/phone-builder/Dockerfile
+4 -3
View File
@@ -1,12 +1,13 @@
FROM quay.io/mozilla/builder:0.5.2
FROM quay.io/mozilla/builder:0.5.4
MAINTAINER Wander Lairson Costa <wcosta@mozilla.com>
# Add utilities and configuration
ADD bin /home/worker/bin
ADD config /home/worker/.aws/config
ADD system-setup.sh /tmp/system-setup.sh
RUN /tmp/system-setup.sh
RUN yum install -y bc lzop
RUN pip install awscli
RUN npm install -g bower gulp apm grunt-cli
# Set a default command useful for debugging
ENTRYPOINT ["validate_task.py"]
testing/docker/phone-builder/VERSION
+1 -1
View File
@@ -1 +1 @@
0.0.11
0.0.13
testing/docker/phone-builder/system-setup.sh
-9
View File
@@ -1,9 +0,0 @@
#!/bin/sh
pip install awscli
# Necessary for dolhin kernel building
yum install -y bc
# Remove ourselves
rm -f $0
testing/docker/tester-device/Dockerfile
+1 -1
View File
@@ -31,7 +31,7 @@ RUN git config --global user.email "mozilla@example.com" && \
# Get node packages
RUN npm install -g taskcluster-vcs@2.3.0
RUN npm install -g taskcluster-vcs@2.3.1
WORKDIR /home/worker
testing/docker/tester-device/bin/entrypoint
+11 -1
View File
@@ -23,4 +23,14 @@ echo "Retrieved device. Session: $SESSION_ID"
curl -o /home/worker/data/device.json -s -H "Accept: application/json" http://$CLOUD_HOST/device/properties
eval $@
buildbot_step 'Clone gaia' tc-vcs checkout /home/worker/gaia/source $GAIA_BASE_REPOSITORY $GAIA_HEAD_REPOSITORY $GAIA_REV $GAIA_REF
cd gaia/source/tests/python/gaia-ui-tests/
python setup.py develop
pip install -Ur gaiatest/tests/requirements.txt
cd /home/worker/
buildbot_step 'Running tests' eval $@
echo "Releasing device."
curl --request POST http://$CLOUD_HOST/device/release
testing/docker/tester/REGISTRY
+1
View File
@@ -0,0 +1 @@
taskcluster
testing/luciddream/README.md
+59
View File
@@ -0,0 +1,59 @@
Luciddream is a test harness for running tests between a Firefox browser and another device, such as a Firefox OS emulator.
The primary goal of the project is to be able to test the Firefox Developer Tools' [remote debugging feature](https://developer.mozilla.org/en-US/docs/Tools/Remote_Debugging), where the developer tools can be used to debug web content running in another browser or device. Mozilla currently doesn't have any automated testing of this feature, so getting some automated tests running is a high priority.
The first planned milestone (for Q4 2014) is to stand up a prototype of a harness, in the process figuring out what the harness will look like and what tests will look like. This work is tracked in [bug 1064253](https://bugzilla.mozilla.org/show_bug.cgi?id=1064253) and is nearing completion. The current harness is based on the [Marionette Test Runner](https://developer.mozilla.org/en-US/docs/Marionette_Test_Runner), and tests right now are a subclass of [Marionette Python tests](https://developer.mozilla.org/en-US/docs/Mozilla/QA/Marionette/Marionette_Python_Tests).
The second planned milestone (for Q1 2015) is to get the harness running in Mozilla's continuous integration environment. Whether this will be in the legacy Buildbot environment or the new TaskCluster environment is yet to be determined. The bare minimum for this milestone will be having the tests run per-checkin against a Firefox Linux desktop build and a Firefox OS emulator both built from the same changeset. A stretch goal for this milestone will be to get tests running against a Firefox Linux desktop build per-checkin paired with stable release builds of the Firefox OS emulator, to be able to test backwards-compatibility of remote debugging. It's likely that as part of this work the repository of record will move from GitHub to mozilla-central, and the harness may be renamed from the current codename to a more descriptive (but less fun) name.
Future directions will likely include testing Firefox desktop remote debugging against other platforms, such as Firefox for Android, desktop Chrome, Chrome for Android, and Safari on iOS.
Points of Contact
=================
The primary developer of this project is Ted Mielczarek (@luser), ted on irc.mozilla.org, :ted in bugzilla.mozilla.org.
The primary DevTools point of contact is Alexandre Poirot (@ochameau), ochameau on irc.mozilla.org, :ochameau in bugzilla.mozilla.org.
Installation and Configuration
==============================
Currently running Luciddream is only supported on Linux, as the Firefox OS emulator is only well-supported there.
Install this module and its Python prerequisites in a virtualenv:
```
virtualenv ./ve
. ./ve/bin/activate
python setup.py develop
```
[Download a Firefox build](http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/latest-mozilla-central/) (if you don't already have one).
Download one of:
* [A Firefox OS emulator build](http://pvtbuilds.pvt.build.mozilla.org/pub/mozilla.org/b2g/tinderbox-builds/mozilla-central-emulator/) (if you don't already have one, this link requires Mozilla VPN access).
* [A B2G desktop build](http://ftp.mozilla.org/pub/mozilla.org/b2g/nightly/latest-mozilla-central/)
Unzip both your Firefox and your Firefox OS emulator/B2G desktop somewhere.
If you're on a 64-bit Ubuntu, you may need to do some fiddling to ensure you have the 32-bit OpenGL libraries available. See the "Solution : have both 32bit and 64bit OpenGL libs installed, with the right symlinks" section [in this blog post](http://rishav006.wordpress.com/2014/05/19/how-to-build-b2g-emulator-in-linux-environment/).
Running Tests
=============
To run with a Firefox OS emulator:
```
runluciddream --b2gpath /path/to/b2g-distro/ --browser-path /path/to/firefox/firefox example-tests/luciddream.ini
```
To run with B2G desktop:
```
runluciddream --b2g-desktop-path /path/to/b2g/b2g --browser-path /path/to/firefox/firefox example-tests/luciddream.ini
```
If you're using a locally-built B2G desktop build which doesn't have a Gaia profile included you should get Gaia and build a profile, and then pass that in with the `--gaia-profile` option:
```
runluciddream --b2g-desktop-path /path/to/obj-b2g/dist/bin/b2g --gaia-profile /path/to/gaia/profile --browser-path /path/to/firefox/firefox example-tests/luciddream.ini
```
testing/luciddream/example-tests/luciddream.ini
+1
View File
@@ -0,0 +1 @@
[test_sample.py]
testing/luciddream/example-tests/test.js
+6
View File
@@ -0,0 +1,6 @@
ok(true, "Assertion from a JS script!");
//ok(false, "test failure");
setTimeout(function() {
ok(true, "Assertion from setTimeout!");
finish();
}, 15);
testing/luciddream/example-tests/test_sample.py
+15
View File
@@ -0,0 +1,15 @@
# Any copyright is dedicated to the Public Domain.
# http://creativecommons.org/publicdomain/zero/1.0/
from luciddream import LucidDreamTestCase
class TestSample(LucidDreamTestCase):
def test_sample(self):
#TODO: make this better
self.assertIsNotNone(self.marionette.session)
self.assertIsNotNone(self.browser.session)
def test_js(self):
'Test that we can run a JavaScript test in both Marionette instances'
self.run_js_test('test.js', self.marionette)
self.run_js_test('test.js', self.browser)
testing/luciddream/luciddream/__init__.py
+111
View File
@@ -0,0 +1,111 @@
#
# This Source Code Form is subject to the terms of the Mozilla Public
# License, v. 2.0. If a copy of the MPL was not distributed with this
# file, You can obtain one at http://mozilla.org/MPL/2.0/.
#
import os
import sys
from marionette.marionette_test import MarionetteTestCase, MarionetteJSTestCase
from marionette_driver.errors import ScriptTimeoutException
class LucidDreamTestCase(MarionetteTestCase):
def __init__(self, marionette_weakref, browser=None, logger=None, **kwargs):
self.browser = browser
self.logger = logger
MarionetteTestCase.__init__(self, marionette_weakref, **kwargs)
def run_js_test(self, filename, marionette):
'''
Run a JavaScript test file and collect its set of assertions
into the current test's results.
:param filename: The path to the JavaScript test file to execute.
May be relative to the current script.
:param marionette: The Marionette object in which to execute the test.
'''
caller_file = sys._getframe(1).f_globals.get('__file__', '')
caller_file = os.path.abspath(caller_file)
script = os.path.join(os.path.dirname(caller_file), filename)
self.assert_(os.path.exists(script), 'Script "%s" must exist' % script)
if hasattr(MarionetteTestCase, 'run_js_test'):
return MarionetteTestCase.run_js_test(self, script, marionette)
#XXX: copy/pasted from marionette_test.py, refactor this!
f = open(script, 'r')
js = f.read()
args = []
head_js = MarionetteJSTestCase.head_js_re.search(js);
if head_js:
head_js = head_js.group(3)
head = open(os.path.join(os.path.dirname(script), head_js), 'r')
js = head.read() + js;
context = MarionetteJSTestCase.context_re.search(js)
if context:
context = context.group(3)
else:
context = 'content'
marionette.set_context(context)
if context != "chrome":
marionette.navigate('data:text/html,<html>test page</html>')
timeout = MarionetteJSTestCase.timeout_re.search(js)
if timeout:
timeout = timeout.group(3)
marionette.set_script_timeout(timeout)
inactivity_timeout = MarionetteJSTestCase.inactivity_timeout_re.search(js)
if inactivity_timeout:
inactivity_timeout = inactivity_timeout.group(3)
try:
results = marionette.execute_js_script(js,
args,
special_powers=True,
inactivity_timeout=inactivity_timeout,
filename=os.path.basename(script))
self.assertTrue(not 'timeout' in script,
'expected timeout not triggered')
if 'fail' in script:
self.assertTrue(len(results['failures']) > 0,
"expected test failures didn't occur")
else:
for failure in results['failures']:
diag = "" if failure.get('diag') is None else failure['diag']
name = "got false, expected true" if failure.get('name') is None else failure['name']
self.logger.test_status(self.test_name, name, 'FAIL',
message=diag)
for failure in results['expectedFailures']:
diag = "" if failure.get('diag') is None else failure['diag']
name = "got false, expected false" if failure.get('name') is None else failure['name']
self.logger.test_status(self.test_name, name, 'FAIL',
expected='FAIL', message=diag)
for failure in results['unexpectedSuccesses']:
diag = "" if failure.get('diag') is None else failure['diag']
name = "got true, expected false" if failure.get('name') is None else failure['name']
self.logger.test_status(self.test_name, name, 'PASS',
expected='FAIL', message=diag)
self.assertEqual(0, len(results['failures']),
'%d tests failed' % len(results['failures']))
if len(results['unexpectedSuccesses']) > 0:
raise _UnexpectedSuccess('')
if len(results['expectedFailures']) > 0:
raise _ExpectedFailure((AssertionError, AssertionError(''), None))
self.assertTrue(results['passed']
+ len(results['failures'])
+ len(results['expectedFailures'])
+ len(results['unexpectedSuccesses']) > 0,
'no tests run')
except ScriptTimeoutException:
if 'timeout' in script:
# expected exception
pass
else:
self.loglines = marionette.get_logs()
raise
testing/luciddream/luciddream/runluciddream.py
+151
View File
@@ -0,0 +1,151 @@
#!/usr/bin/env python
#
# This Source Code Form is subject to the terms of the Mozilla Public
# License, v. 2.0. If a copy of the MPL was not distributed with this
# file, You can obtain one at http://mozilla.org/MPL/2.0/.
#
from __future__ import print_function
import argparse
import os
import sys
from luciddream import LucidDreamTestCase
from marionette import Marionette
from marionette.runner import BaseMarionetteTestRunner
import marionette
from mozlog import structured
class CommandLineError(Exception):
pass
def validate_options(options):
if not (options.emulator_path or options.b2g_desktop_path):
raise CommandLineError('You must specify --emulator-path or ' +
'--b2g-desktop-path')
if options.emulator_path and options.b2g_desktop_path:
raise CommandLineError('You may only use one of --emulator-path or ' +
'--b2g-desktop-path')
if options.gaia_profile and options.emulator_path:
raise CommandLineError('You may not use --gaia-profile with ' +
'--emulator-path')
if not options.browser_path:
raise CommandLineError('You must specify --browser-path')
if not os.path.isfile(options.manifest):
raise CommandLineError('The manifest at "%s" does not exist!'
% options.manifest)
# BaseMarionetteOptions has a lot of stuff we don't care about, and
# it seems hard to apply directly to this problem. We can revisit this
# decision later if necessary.
def parse_args(in_args):
parser = argparse.ArgumentParser(description='Run Luciddream tests.')
parser.add_argument('--emulator-arch', dest='emulator_arch', action='store',
default='arm',
help='Architecture of emulator to use: x86 or arm')
parser.add_argument('--emulator-path', dest='emulator_path', action='store',
help='path to B2G repo or qemu dir')
parser.add_argument('--b2g-desktop-path', dest='b2g_desktop_path',
action='store',
help='path to B2G desktop binary')
parser.add_argument('--browser-path', dest='browser_path', action='store',
help='path to Firefox binary')
parser.add_argument('--gaia-profile', dest='gaia_profile', action='store',
help='path to Gaia profile')
parser.add_argument('manifest', metavar='MANIFEST', action='store',
help='path to manifest of tests to run')
structured.commandline.add_logging_group(parser)
args = parser.parse_args(in_args)
try:
validate_options(args)
return args
except CommandLineError as e:
print('Error: ', e.args[0], file=sys.stderr)
parser.print_help()
raise
class LucidDreamTestRunner(BaseMarionetteTestRunner):
def __init__(self, **kwargs):
BaseMarionetteTestRunner.__init__(self, **kwargs)
#TODO: handle something like MarionetteJSTestCase
self.test_handlers = [LucidDreamTestCase]
def start_browser(browser_path, app_args):
'''
Start a Firefox browser and return a Marionette instance that
can talk to it.
'''
marionette = Marionette(
bin=browser_path,
# Need to avoid the browser and emulator's ports stepping
# on each others' toes.
port=2929,
app_args=app_args,
gecko_log="firefox.log"
)
runner = marionette.runner
if runner:
runner.start()
marionette.wait_for_port()
marionette.start_session()
marionette.set_context(marionette.CONTEXT_CHROME)
return marionette
#TODO: make marionette/client/marionette/runtests.py importable so we can
# just use cli from there. A lot of this is copy/paste from that function.
def run(browser_path=None, b2g_desktop_path=None, emulator_path=None, emulator_arch=None, gaia_profile=None, manifest=None, browser_args=None, **kwargs):
# It's sort of debatable here whether the marionette instance managed
# by the test runner should be the browser or the emulator. Right now
# it's the emulator because it feels like there's more fiddly setup around
# that, but longer-term if we want to run tests against different
# (non-B2G) targets this won't match up very well, so maybe it ought to
# be the browser?
browser = start_browser(browser_path, browser_args)
kwargs["browser"] = browser
if not "logger" in kwargs:
logger = structured.commandline.setup_logging(
"luciddream", kwargs, {"tbpl": sys.stdout})
kwargs["logger"] = logger
if emulator_path:
kwargs['homedir'] = emulator_path
kwargs['emulator'] = emulator_arch
elif b2g_desktop_path:
# Work around bug 859952
if '-bin' not in b2g_desktop_path:
if b2g_desktop_path.endswith('.exe'):
newpath = b2g_desktop_path[:-4] + '-bin.exe'
else:
newpath = b2g_desktop_path + '-bin'
if os.path.exists(newpath):
b2g_desktop_path = newpath
kwargs['binary'] = b2g_desktop_path
kwargs['app'] = 'b2gdesktop'
if gaia_profile:
kwargs['profile'] = gaia_profile
else:
kwargs['profile'] = os.path.join(
os.path.dirname(b2g_desktop_path),
'gaia',
'profile'
)
runner = LucidDreamTestRunner(**kwargs)
runner.run_tests([manifest])
if runner.failed > 0:
sys.exit(10)
sys.exit(0)
def main():
args = parse_args(sys.argv[1:])
run(**vars(args))
if __name__ == '__main__':
main()
testing/luciddream/mach_commands.py
+95
View File
@@ -0,0 +1,95 @@
# This Source Code Form is subject to the terms of the Mozilla Public
# License, v. 2.0. If a copy of the MPL was not distributed with this
# file, You can obtain one at http://mozilla.org/MPL/2.0/.
# Integrates luciddream test runner with mach.
import os
from mozbuild.base import (
MachCommandBase,
MachCommandConditions as conditions,
MozbuildObject,
)
from mach.decorators import (
CommandArgument,
CommandProvider,
Command,
)
class LucidDreamRunner(MozbuildObject):
"""Run luciddream tests."""
def run_tests(self, **kwargs):
self._run_make(target='jetpack-tests')
@CommandProvider
class MachCommands(MachCommandBase):
@Command('luciddream', category='testing',
description='Run the luciddream test suite (remote debugging).')
@CommandArgument("--consoles", action="store_true",
help="Open jsconsole in both runtimes.")
@CommandArgument('--b2g-desktop', dest="b2g_desktop_path", type=str, default=None,
help='Path to b2g desktop binary.')
@CommandArgument('--browser', dest="browser_path", type=str, default=None,
help='Path to firefox binary.')
@CommandArgument('--gaia-profile', type=str, default=None,
help='Path to gaia profile, optional, if not bundled with b2g desktop.')
@CommandArgument('--emulator', dest="emulator_path", type=str, default=None,
help='Path to android emulator.')
@CommandArgument('--emulator-arch', type=str, default="arm",
help='Emulator arch: x86 or arm.')
@CommandArgument('test_paths', default=None, nargs='*', metavar='TEST',
help='Test to run. Can be specified as a single file, a '
'directory, or omitted. If omitted, the entire test suite is '
'executed.')
def run_luciddream_test(self, browser_path, b2g_desktop_path, test_paths, consoles, **params):
# import luciddream lazily as its marionette dependency make ./mach clobber fails
# early on TBPL
import luciddream.runluciddream
# get_binary_path is going to throw if we haven't built any product
# but luciddream can still be run if we provide both binaries...
binary_path=False
try:
binary_path = self.get_binary_path()
except Exception:
pass
# otherwise, if we have a build, automatically fetch the binary
if conditions.is_b2g(self):
if not b2g_desktop_path and binary_path:
b2g_desktop_path = binary_path
else:
if not browser_path and binary_path:
browser_path = binary_path
if not browser_path:
print "Need firefox binary path via --browser_path argument"
return 1
elif not os.path.exists(browser_path):
print "Firefox binary doesn't exists: " + browser_path
return 1
if not b2g_desktop_path:
print "Need b2g desktop binary path via --b2g-desktop argument"
return 1
elif not os.path.exists(b2g_desktop_path):
print "B2G desktop binary doesn't exists: " + b2g_desktop_path
return 1
if not test_paths or len(test_paths) == 0:
print "Please specify a test manifest to run"
return 1
browser_args = None
if consoles:
browser_args = ["-jsconsole"]
if "app_args" in params and isinstance(params["app_args"], list):
params["app_args"].append("-jsconsole")
else:
params["app_args"] = ["-jsconsole"]
for test in test_paths:
luciddream.runluciddream.run(browser_path=browser_path, b2g_desktop_path=b2g_desktop_path,
manifest=test, browser_args=browser_args, **params)
testing/luciddream/requirements.txt
+2
View File
@@ -0,0 +1,2 @@
marionette-client>=0.8.5
mozlog
testing/luciddream/setup.py
+43
View File
@@ -0,0 +1,43 @@
#!/usr/bin/env python
#
# This Source Code Form is subject to the terms of the Mozilla Public
# License, v. 2.0. If a copy of the MPL was not distributed with this file,
# You can obtain one at http://mozilla.org/MPL/2.0/.
#
import os
from setuptools import setup, find_packages
try:
here = os.path.dirname(os.path.abspath(__file__))
description = file(os.path.join(here, 'README.md')).read()
except IOError:
description = ''
version = '0.1'
dependencies = open('requirements.txt', 'r').read().splitlines()
setup(
name='luciddream',
version=version,
description='''
Luciddream is a test harness for running tests between
a Firefox browser and another device, such as a Firefox OS
emulator.
''',
long_description=description,
classifiers=[], # Get strings from http://www.python.org/pypi?%3Aaction=list_classifiers
author='Ted Mielczarek',
author_email='ted@mielczarek.org',
url='',
license='MPL 2.0',
packages=find_packages(exclude=['ez_setup', 'examples', 'tests']),
include_package_data=True,
package_data={'': ['*.js', '*.css', '*.html', '*.txt', '*.xpi', '*.rdf', '*.xul', '*.jsm', '*.xml'],},
zip_safe=False,
install_requires=dependencies,
entry_points = {
'console_scripts': ['runluciddream=luciddream.runluciddream:main'],
}
)
testing/mach_commands.py
+4
View File
@@ -89,6 +89,10 @@ TEST_SUITES = {
'mach_command': 'mochitest',
'kwargs': {'flavor': 'plain', 'test_paths': None},
},
'luciddream': {
'mach_command': 'luciddream',
'kwargs': {'test_paths': None},
},
'reftest': {
'aliases': ('RR', 'rr', 'Rr'),
'mach_command': 'reftest',
testing/marionette/mach_commands.py
-1
View File
@@ -4,7 +4,6 @@
from __future__ import unicode_literals
import imp
import os
import sys
import argparse
testing/taskcluster/mach_commands.py
+31 -186
View File
@@ -5,11 +5,8 @@
# file, You can obtain one at http://mozilla.org/MPL/2.0/.
import os
import os.path
import json
import copy
import datetime
import subprocess
import sys
import urllib2
@@ -30,12 +27,13 @@ import taskcluster_graph.build_task
ROOT = os.path.dirname(os.path.realpath(__file__))
GECKO = os.path.realpath(os.path.join(ROOT, '..', '..'))
DOCKER_ROOT = os.path.join(ROOT, '..', 'docker')
MOZHARNESS_CONFIG = os.path.join(GECKO, 'testing', 'mozharness', 'mozharness.json')
# XXX: If/when we have the taskcluster queue use construct url instead
ARTIFACT_URL = 'https://queue.taskcluster.net/v1/task/{}/artifacts/{}'
REGISTRY = open(os.path.join(DOCKER_ROOT, 'REGISTRY')).read().strip()
DEFINE_TASK = 'queue:define-task:aws-provisioner/{}'
DEFINE_TASK = 'queue:define-task:aws-provisioner-v1/{}'
TREEHERDER_ROUTE_PREFIX = 'tc-treeherder-stage'
TREEHERDER_ROUTES = {
@@ -45,45 +43,16 @@ TREEHERDER_ROUTES = {
DEFAULT_TRY = 'try: -b do -p all -u all'
DEFAULT_JOB_PATH = os.path.join(
ROOT, 'tasks', 'branches', 'mozilla-central', 'job_flags.yml'
ROOT, 'tasks', 'branches', 'base_jobs.yml'
)
def get_hg_url():
''' Determine the url for the mercurial repository'''
try:
url = subprocess.check_output(
['hg', 'path', 'default'],
stderr=subprocess.PIPE
)
except subprocess.CalledProcessError:
sys.stderr.write(
"Error: Could not determine the current hg repository url. " \
"Ensure command is executed within a hg respository"
)
sys.exit(1)
return url
def get_latest_hg_revision(repository):
''' Retrieves the revision number of the latest changed head'''
try:
revision = subprocess.check_output(
['hg', 'id', '-r', 'tip', repository, '-i'],
stderr=subprocess.PIPE
).strip('\n')
except subprocess.CalledProcessError:
sys.stderr.write(
"Error: Could not determine the latest hg revision at {} " \
"Ensure command is executed within a cloned hg respository and " \
"remote default remote repository is accessible".format(repository)
)
sys.exit(1)
return revision
def load_mozharness_info():
with open(MOZHARNESS_CONFIG) as content:
return json.load(content)
def docker_image(name):
''' Determine the docker tag/revision from an in tree docker file '''
repository_path = os.path.join(DOCKER_ROOT, name, 'REPOSITORY')
repository_path = os.path.join(DOCKER_ROOT, name, 'REGISTRY')
repository = REGISTRY
version = open(os.path.join(DOCKER_ROOT, name, 'VERSION')).read().strip()
@@ -100,7 +69,7 @@ def get_task(task_id):
def gaia_info():
'''
Fetch details from in tree gaia.json (which links this version of
goanna->gaia) and construct the usual base/head/ref/rev pairing...
gecko->gaia) and construct the usual base/head/ref/rev pairing...
'''
gaia = json.load(open(os.path.join(GECKO, 'b2g', 'config', 'gaia.json')))
@@ -183,7 +152,7 @@ class DecisionTask(object):
'owner': params['owner'],
'as_slugid': SlugidJar(),
'from_now': json_time_from_now,
'now': datetime.datetime.now().isoformat()
'now': current_json_time()
}.items())
task = templates.load(params['task'], parameters)
print(json.dumps(task, indent=4))
@@ -195,9 +164,6 @@ class Graph(object):
@CommandArgument('--base-repository',
default=os.environ.get('GECKO_BASE_REPOSITORY'),
help='URL for "base" repository to clone')
@CommandArgument('--mozharness-repository',
default='https://hg.mozilla.org/build/mozharness',
help='URL for custom mozharness repo')
@CommandArgument('--head-repository',
default=os.environ.get('GECKO_HEAD_REPOSITORY'),
help='URL for "head" repository to fetch revision from')
@@ -207,12 +173,6 @@ class Graph(object):
@CommandArgument('--head-rev',
default=os.environ.get('GECKO_HEAD_REV'),
help='Commit revision to use from head repository')
@CommandArgument('--mozharness-rev',
default='default',
help='Commit revision to use from mozharness repository')
@CommandArgument('--mozharness-ref',
default='master',
help='Commit ref to use from mozharness repository')
@CommandArgument('--message',
help='Commit message to be parsed. Example: "try: -b do -p all -u all"')
@CommandArgument('--revision-hash',
@@ -249,6 +209,8 @@ class Graph(object):
jobs = templates.load(job_path, {})
job_graph = parse_commit(message, jobs)
mozharness = load_mozharness_info()
# Template parameters used when expanding the graph
parameters = dict(gaia_info().items() + {
'project': project,
@@ -261,10 +223,10 @@ class Graph(object):
'head_rev': params['head_rev'],
'owner': params['owner'],
'from_now': json_time_from_now,
'now': datetime.datetime.now().isoformat(),
'mozharness_repository': params['mozharness_repository'],
'mozharness_rev': params['mozharness_rev'],
'mozharness_ref': params['mozharness_ref'],
'now': current_json_time(),
'mozharness_repository': mozharness['repo'],
'mozharness_rev': mozharness['revision'],
'mozharness_ref':mozharness.get('reference', mozharness['revision']),
'revision_hash': params['revision_hash']
}.items())
@@ -429,9 +391,6 @@ class CIBuild(object):
description="Create taskcluster try server build task")
@CommandArgument('--base-repository',
help='URL for "base" repository to clone')
@CommandArgument('--mozharness-repository',
default='http://hg.mozilla.org/build/mozharness',
help='URL for custom mozharness repo')
@CommandArgument('--head-repository',
required=True,
help='URL for "head" repository to fetch revision from')
@@ -440,14 +399,12 @@ class CIBuild(object):
@CommandArgument('--head-rev',
required=True,
help='Commit revision to use')
@CommandArgument('--mozharness-repository',
help='URL for custom mozharness repo')
@CommandArgument('--mozharness-rev',
default='tip',
help='Commit revision to use from mozharness repository')
@CommandArgument('--mozharness-ref',
default='master',
help='Commit ref to use from mozharness repository')
@CommandArgument('--owner',
required=True,
default='foobar@mozilla.com',
help='email address of who owns this graph')
@CommandArgument('build_task',
help='path to build task definition')
@@ -464,6 +421,16 @@ class CIBuild(object):
head_ref = params['head_ref'] or head_rev
mozharness = load_mozharness_info()
mozharness_repo = params['mozharness_repository']
if mozharness_repo is None:
mozharness_repo = mozharness['repo']
mozharness_rev = params['mozharness_rev']
if mozharness_rev is None:
mozharness_rev = mozharness['revision']
build_parameters = dict(gaia_info().items() + {
'docker_image': docker_image,
'owner': params['owner'],
@@ -473,9 +440,9 @@ class CIBuild(object):
'head_repository': head_repository,
'head_rev': head_rev,
'head_ref': head_ref,
'mozharness_repository': params['mozharness_repository'],
'mozharness_ref': params['mozharness_ref'],
'mozharness_rev': params['mozharness_rev']
'mozharness_repository': mozharness_repo,
'mozharness_ref': mozharness_rev,
'mozharness_rev': mozharness_rev
}.items())
try:
@@ -490,125 +457,3 @@ class CIBuild(object):
taskcluster_graph.build_task.validate(build_task)
print(json.dumps(build_task['task'], indent=4))
@CommandProvider
class CITest(object):
@Command('taskcluster-test', category='ci',
description='Create taskcluster try server test task')
@CommandArgument('--task-id',
help='the task id to pick the correct build and tests')
@CommandArgument('--total-chunks', type=int,
help='total number of chunks')
@CommandArgument('--chunk', type=int,
help='current chunk')
@CommandArgument('--owner',
help='email address of who owns this graph')
@CommandArgument('test_task',
help='path to the test task definition')
def create_ci_test(self, test_task, task_id='', total_chunks=1, chunk=1, owner=''):
if total_chunks is None:
total_chunks = 1
if chunk is None:
chunk = 1
if chunk < 1 or chunk > total_chunks:
raise ValueError(
'"chunk" must be a value between 1 and "total_chunks (default 1)"')
build_url, img_url, tests_url = self._get_build_and_tests_url(task_id)
test_parameters = dict(gaia_info().items() + {
'docker_image': docker_image,
'build_url': ARTIFACT_URL.format(task_id, build_url),
'img_url': ARTIFACT_URL.format(task_id, img_url),
'tests_url': ARTIFACT_URL.format(task_id, tests_url),
'total_chunks': total_chunks,
'chunk': chunk,
'owner': owner,
'from_now': json_time_from_now,
'now': current_json_time()
}.items())
try:
templates = Templates(ROOT)
test_task = templates.load(test_task, test_parameters)
except IOError:
sys.stderr.write(
"Could not load test task file. Ensure path is a relative " \
"path from testing/taskcluster"
)
sys.exit(1)
print(json.dumps(test_task['task'], indent=4))
def _get_build_and_tests_url(self, task_id):
task = get_task(task_id)
locations = task['extra']['locations']
return locations['build'], locations.get('img', ''), locations['tests']
@CommandProvider
class CIDockerRun(object):
@Command('taskcluster-docker-run', category='ci',
description='Run a docker image and optionally mount local hg repos. ' \
'Repos will be mounted to /home/worker/x/source accordingly. ' \
'For example, to run a centos image and mount local goanna ' \
'and gaia repos: mach ci-docker-run --local-goanna-repo ' \
'/home/user/mozilla-central/ --local-gaia-repo /home/user/gaia/ '\
'--docker-flags="-t -i" centos:centos7 /bin/bash')
@CommandArgument('--local-goanna-repo',
action='store', dest='local_goanna_repo',
help='local goanna hg repository for volume mount')
@CommandArgument('--goanna-revision',
action='store', dest='goanna_revision',
help='local goanna repo revision (defaults to latest)')
@CommandArgument('--local-gaia-repo',
action='store', dest='local_gaia_repo',
help='local gaia hg repository for volume mount')
@CommandArgument('--mozconfig',
help='The mozconfig file for building goanna')
@CommandArgument('--docker-flags',
action='store', dest='flags',
help='string of run flags (i.e. --docker-flags="-i -t")')
@CommandArgument('image',
help='name of docker image to run')
@CommandArgument('command',
nargs='*',
help='command to run inside the docker image')
def ci_docker_run(self, local_goanna_repo='', goanna_revision='',
local_gaia_repo='', mozconfig="", flags="", **kwargs):
''' Run docker image and optionally volume mount specified local repos '''
goanna_mount_point='/home/worker/mozilla-central/source/'
gaia_mount_point='/home/worker/gaia/source/'
cmd_out = ['docker', 'run']
if flags:
cmd_out.extend(flags.split())
if local_goanna_repo:
if not os.path.exists(local_goanna_repo):
print("Gecko repository path doesn't exist: %s" % local_goanna_repo)
sys.exit(1)
if not goanna_revision:
goanna_revision = get_latest_hg_revision(local_goanna_repo)
cmd_out.extend(['-v', '%s:%s' % (local_goanna_repo, goanna_mount_point)])
cmd_out.extend(['-e', 'REPOSITORY=%s' % goanna_mount_point])
cmd_out.extend(['-e', 'REVISION=%s' % goanna_revision])
if local_gaia_repo:
if not os.path.exists(local_gaia_repo):
print("Gaia repository path doesn't exist: %s" % local_gaia_repo)
sys.exit(1)
cmd_out.extend(['-v', '%s:%s' % (local_gaia_repo, gaia_mount_point)])
cmd_out.extend(['-e', 'GAIA_REPOSITORY=%s' % gaia_mount_point])
if mozconfig:
cmd_out.extend(['-e', 'MOZCONFIG=%s' % mozconfig])
cmd_out.append(kwargs['image'])
for cmd_x in kwargs['command']:
cmd_out.append(cmd_x)
try:
subprocess.check_call(cmd_out)
except subprocess.CalledProcessError:
sys.stderr.write("Docker run command returned non-zero status. Attempted:\n")
cmd_line = ''
for x in cmd_out:
cmd_line = cmd_line + x + ' '
sys.stderr.write(cmd_line + '\n')
sys.exit(1)
testing/taskcluster/scripts/phone-builder/build-dolphin.sh
+66
View File
@@ -0,0 +1,66 @@
#! /bin/bash -vex
# Ensure all the scripts in this dir are on the path....
DIRNAME=$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )
PATH=$DIRNAME:$PATH
WORKSPACE=$1
### Check that require variables are defined
test -d $WORKSPACE
test $GECKO_HEAD_REPOSITORY # Should be an hg repository url to pull from
test $GECKO_BASE_REPOSITORY # Should be an hg repository url to clone from
test $GECKO_HEAD_REV # Should be an hg revision to pull down
test $MOZHARNESS_REPOSITORY # mozharness repository
test $MOZHARNESS_REV # mozharness revision
test $TARGET
test $VARIANT
. ../builder/setup-ccache.sh
# First check if the mozharness directory is available. This is intended to be
# used locally in development to test mozharness changes:
#
# $ docker -v your_mozharness:/home/worker/mozharness ...
#
if [ ! -d mozharness ]; then
tc-vcs checkout mozharness $MOZHARNESS_REPOSITORY $MOZHARNESS_REPOSITORY $MOZHARNESS_REV
fi
# Figure out where the remote manifest is so we can use caches for it.
MANIFEST=$(repository-url.py $GECKO_HEAD_REPOSITORY $GECKO_HEAD_REV b2g/config/$TARGET/sources.xml)
tc-vcs repo-checkout $WORKSPACE/B2G https://git.mozilla.org/b2g/B2G.git $MANIFEST
# Ensure symlink has been created to gecko...
rm -f $WORKSPACE/B2G/gecko
ln -s $WORKSPACE/gecko $WORKSPACE/B2G/gecko
debug_flag=""
if [ 0$B2G_DEBUG -ne 0 ]; then
debug_flag='--debug'
fi
./mozharness/scripts/b2g_build.py \
--config b2g/taskcluster-phone.py \
"$debug_flag" \
--disable-mock \
--variant=$VARIANT \
--work-dir=$WORKSPACE/B2G \
--gaia-languages-file $WORKSPACE/B2G/device/sprd/scx15/languages.json \
--log-level=debug \
--target=$TARGET \
--b2g-config-dir=$TARGET \
--checkout-revision=$GECKO_HEAD_REV \
--base-repo=$GECKO_BASE_REPOSITORY \
--repo=$GECKO_HEAD_REPOSITORY
# Move files into artifact locations!
mkdir -p $HOME/artifacts
mv $WORKSPACE/B2G/upload/sources.xml $HOME/artifacts/sources.xml
mv $WORKSPACE/B2G/upload/b2g-*.crashreporter-symbols.zip $HOME/artifacts/b2g-crashreporter-symbols.zip
mv $WORKSPACE/B2G/upload/b2g-*.android-arm.tar.gz $HOME/artifacts/b2g-android-arm.tar.gz
mv $WORKSPACE/B2G/upload/${TARGET}.zip $HOME/artifacts/${TARGET}.zip
mv $WORKSPACE/B2G/upload/gaia.zip $HOME/artifacts/gaia.zip
ccache -s
testing/taskcluster/scripts/phone-builder/build-phone-nightly.sh → testing/taskcluster/scripts/phone-builder/build-phone-ota.sh
+14 -6
View File
@@ -11,11 +11,20 @@ if [ ! -d $HOME/.ssh ]; then
fi
aws s3 cp s3://b2g-nightly-credentials/balrog_credentials .
aws s3 cp s3://b2g-nightly-credentials/b2g-rsa $HOME/.ssh/
mar_file=b2g-${TARGET%%-*}-gecko-update.mar
# We need different platform names for each variant (user, userdebug and
# eng). We do not append variant suffix for "user" to keep compability with
# verions already installed in the phones.
if [ $VARIANT == "user" ]; then
PLATFORM=$TARGET
else
PLATFORM=$TARGET-$VARIANT
fi
./mozharness/scripts/b2g_build.py \
--config b2g/taskcluster-phone-nightly.py \
--config balrog/staging.py \
--config balrog/docker-worker.py \
"$debug_flag" \
--disable-mock \
--variant=$VARIANT \
@@ -27,18 +36,17 @@ aws s3 cp s3://b2g-nightly-credentials/b2g-rsa $HOME/.ssh/
--checkout-revision=$GECKO_HEAD_REV \
--base-repo=$GECKO_BASE_REPOSITORY \
--repo=$GECKO_HEAD_REPOSITORY \
--platform $TARGET \
--complete-mar-url https://queue.taskcluster.net/v1/task/$TASK_ID/runs/0/artifacts/public/build/b2g-${TARGET%%-*}-gecko-update.mar \
--platform $PLATFORM \
--complete-mar-url https://queue.taskcluster.net/v1/task/$TASK_ID/runs/$RUN_ID/artifacts/public/build/$mar_file
# Don't cache backups
rm -rf $WORKSPACE/B2G/backup-*
rm -f balrog_credentials
rm -f $HOME/.ssh/b2g-rsa
mkdir -p $HOME/artifacts
mkdir -p $HOME/artifacts-public
mv $WORKSPACE/B2G/upload-public/b2g-flame-gecko-update.mar $HOME/artifacts-public/b2g-flame-gecko-update.mar
mv $WORKSPACE/B2G/upload-public/$mar_file $HOME/artifacts-public/
mv $WORKSPACE/B2G/upload/sources.xml $HOME/artifacts/sources.xml
#mv $WORKSPACE/B2G/upload/b2g-*.crashreporter-symbols.zip $HOME/artifacts/b2g-crashreporter-symbols.zip
mv $WORKSPACE/B2G/upload/b2g-*.android-arm.tar.gz $HOME/artifacts/b2g-android-arm.tar.gz
testing/taskcluster/scripts/phone-builder/build-spark-ota.sh
+58
View File
@@ -0,0 +1,58 @@
#!/bin/bash -vex
. pre-build.sh
if [ 0$B2G_DEBUG -ne 0 ]; then
DEBUG_SUFFIX=-debug
fi
if [ $TARGET == "aries" -o $TARGET == "shinano" ]; then
# caching objects might be dangerous for some devices (aka aries)
rm -rf $WORKSPACE/B2G/objdir*
rm -rf $WORKSPACE/B2G/out
fi
aws s3 cp s3://b2g-nightly-credentials/balrog_credentials .
mar_file=b2g-${TARGET%%-*}-gecko-update.mar
# We need different platform names for each variant (user, userdebug and
# eng). We do not append variant suffix for "user" to keep compability with
# verions already installed in the phones.
if [ $VARIANT == "user" ]; then
PLATFORM=$TARGET
else
PLATFORM=$TARGET-$VARIANT
fi
./mozharness/scripts/b2g_lightsaber.py \
--config b2g/taskcluster-lightsaber-nightly.py \
--config balrog/docker-worker.py \
"$debug_flag" \
--disable-mock \
--variant=$VARIANT \
--work-dir=$WORKSPACE/B2G \
--gaia-languages-file locales/languages_all.json \
--log-level=debug \
--target=$TARGET \
--b2g-config-dir=$TARGET \
--checkout-revision=$GECKO_HEAD_REV \
--base-repo=$GECKO_BASE_REPOSITORY \
--repo=$GECKO_HEAD_REPOSITORY \
--platform $PLATFORM \
--complete-mar-url https://queue.taskcluster.net/v1/task/$TASK_ID/runs/$RUN_ID/artifacts/public/build/$mar_file \
# Don't cache backups
rm -rf $WORKSPACE/B2G/backup-*
rm -f balrog_credentials
mkdir -p $HOME/artifacts
mkdir -p $HOME/artifacts-public
mv $WORKSPACE/B2G/upload-public/$mar_file $HOME/artifacts-public/
mv $WORKSPACE/B2G/upload/sources.xml $HOME/artifacts/sources.xml
#mv $WORKSPACE/B2G/upload/b2g-*.crashreporter-symbols.zip $HOME/artifacts/b2g-crashreporter-symbols.zip
mv $WORKSPACE/B2G/upload/b2g-*.android-arm.tar.gz $HOME/artifacts/b2g-android-arm.tar.gz
mv $WORKSPACE/B2G/upload/${TARGET}.zip $HOME/artifacts/${TARGET}.zip
mv $WORKSPACE/B2G/upload/gaia.zip $HOME/artifacts/gaia.zip
ccache -s
testing/taskcluster/scripts/phone-builder/build-spark.sh
+40
View File
@@ -0,0 +1,40 @@
#!/bin/bash -vex
. pre-build.sh
# We need different platform names for each variant (user, userdebug and
# eng). We do not append variant suffix for "user" to keep compability with
# verions already installed in the phones.
if [ $VARIANT == "user" ]; then
PLATFORM=$TARGET
else
PLATFORM=$TARGET-$VARIANT
fi
./mozharness/scripts/b2g_lightsaber.py \
--config b2g/taskcluster-lightsaber.py \
"$debug_flag" \
--disable-mock \
--variant=$VARIANT \
--work-dir=$WORKSPACE/B2G \
--gaia-languages-file locales/languages_all.json \
--log-level=debug \
--target=$TARGET \
--b2g-config-dir=$TARGET \
--checkout-revision=$GECKO_HEAD_REV \
--base-repo=$GECKO_BASE_REPOSITORY \
--repo=$GECKO_HEAD_REPOSITORY
# Don't cache backups
rm -rf $WORKSPACE/B2G/backup-*
# Move files into artifact locations!
mkdir -p $HOME/artifacts
mv $WORKSPACE/B2G/upload/sources.xml $HOME/artifacts/sources.xml
mv $WORKSPACE/B2G/upload/b2g-*.crashreporter-symbols.zip $HOME/artifacts/b2g-crashreporter-symbols.zip
mv $WORKSPACE/B2G/upload/b2g-*.android-arm.tar.gz $HOME/artifacts/b2g-android-arm.tar.gz
mv $WORKSPACE/B2G/upload/${TARGET}.zip $HOME/artifacts/${TARGET}.zip
mv $WORKSPACE/B2G/upload/gaia.zip $HOME/artifacts/gaia.zip
ccache -s
testing/taskcluster/scripts/phone-builder/pre-build.sh
+2 -9
View File
@@ -1,4 +1,4 @@
#! /bin/bash -vex
#!/bin/bash -vex
# Ensure all the scripts in this dir are on the path....
DIRNAME=$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )
@@ -19,14 +19,7 @@ test $VARIANT
. ../builder/setup-ccache.sh
# First check if the mozharness directory is available. This is intended to be
# used locally in development to test mozharness changes:
#
# $ docker -v your_mozharness:/home/worker/mozharness ...
#
if [ ! -d mozharness ]; then
tc-vcs checkout mozharness $MOZHARNESS_REPOSITORY $MOZHARNESS_REPOSITORY $MOZHARNESS_REV $MOZHARNESS_REF
fi
tc-vcs checkout mozharness $MOZHARNESS_REPOSITORY $MOZHARNESS_REPOSITORY $MOZHARNESS_REV $MOZHARNESS_REF
# Figure out where the remote manifest is so we can use caches for it.
MANIFEST=$(repository-url.py $GECKO_HEAD_REPOSITORY $GECKO_HEAD_REV b2g/config/$TARGET/sources.xml)
testing/taskcluster/tasks/branches/b2g-inbound/job_flags.yml
+1 -18
View File
@@ -3,21 +3,4 @@
# see <gecko>/testing/taskcluster/tasks/job_flags.yml
$inherits:
from: tasks/branches/mozilla-central/job_flags.yml
builds:
flame-kk:
platforms:
- b2g
types:
opt:
task: tasks/builds/b2g_flame_kk_opt.yml
flame-kk-eng:
platforms:
- b2g
types:
opt:
task: tasks/builds/b2g_flame_kk_eng.yml
gaia-ui-test-sanity:
task: tasks/tests/flame_kk_gaia_ui_test_sanity.yml
from: tasks/branches/base_jobs.yml
testing/taskcluster/tasks/branches/base_job_flags.yml
+9
View File
@@ -13,8 +13,16 @@ flags:
- linux64-mulet # Firefox desktop - b2g gecko linux 64 bit
- macosx64_gecko # b2g desktop osx 64 bit
- win32_gecko # b2g desktop win 32 bit
- flame-kk-ota
- flame-kk # b2g flame kitkat
- flame-kk-eng # b2g flame eng build
- dolphin
- dolphin-eng
- dolphin-512
- dolphin-512-eng
- aries
- aries-ota
- aries-eng
tests:
- cppunit
@@ -30,6 +38,7 @@ flags:
- gaia-ui-test-oop
- gaia-ui-test-accessibility
- gaia-ui-test-functional
- gaia-ui-test-functional-dsds
- gaia-ui-test-unit
- jetpack
- jittests

Some files were not shown because too many files have changed in this diff Show More