Stratos Dimopoulos

In this paper, we present JUSTICE, a fair-share deadline-aware resource allocator for big data cluster managers. In resource constrained environments, where resource contention introduces significant execution delays, JUSTICE outperforms the popular existing fair-share allocator that is implemented as part of Mesos and YARN. JUSTICE uses deadline information supplied with each job and historical job execution logs to implement admission control. It automatically adapts to changing workload… Show more

In this paper, we present JUSTICE, a fair-share deadline-aware resource allocator for big data cluster managers. In resource constrained environments, where resource contention introduces significant execution delays, JUSTICE outperforms the popular existing fair-share allocator that is implemented as part of Mesos and YARN. JUSTICE uses deadline information supplied with each job and historical job execution logs to implement admission control. It automatically adapts to changing workload conditions to assign enough resources for each job to meet its deadline “just in time.” We use trace-based simulation of production YARN workloads to evaluate JUSTICE under different deadline formulations. We compare JUSTICE to the existing fair-share allocation policy deployed on cluster managers like YARN and Mesos and find that in resource-constrained settings, JUSTICE improves fairness, satisfies significantly more deadlines, and utilizes resources more efficiently. Show less

In this paper, we present PYTHIA, deadline-aware
admission control for systems that execute jobs from multiple big
data (batch) frameworks using shared resources. PYTHIA adds
support for deadline-driven workloads in resource-constrained
cloud settings, for use by resource negotiators such as Apache
Mesos or YARN. PYTHIA uses histories of job statistics to
estimate the minimum number of CPUs to allocate to a job in
order for it to meet its deadline. PYTHIA admits jobs… Show more

In this paper, we present PYTHIA, deadline-aware
admission control for systems that execute jobs from multiple big
data (batch) frameworks using shared resources. PYTHIA adds
support for deadline-driven workloads in resource-constrained
cloud settings, for use by resource negotiators such as Apache
Mesos or YARN. PYTHIA uses histories of job statistics to
estimate the minimum number of CPUs to allocate to a job in
order for it to meet its deadline. PYTHIA admits jobs when
these resources are available. Any job not admitted “fails fast”
and wastes no resources. We implement a PYTHIA prototype
and empirically evaluate it using production YARN traces under
different resource constraints and deadline assignments. Our
results show that PYTHIA is able to meet significantly more
deadlines than fair share approaches and wastes fewer cloud
resources in resource-limited scenarios, for the workloads, cluster
sizes, and deadline assignments that we consider Show less

In this paper, we characterize the behavior of “big”
and “fast” data analysis frameworks, in multi-tenant, shared
settings for which computing resources (CPU and memory)
are limited, an increasingly common scenario used to increase
utilization and lower cost. We study how popular analytics
frameworks behave and interfere with each other under such
constraints. We empirically evaluate Hadoop, Spark, and Storm
multi-tenant workloads managed by Mesos. Our results show
that… Show more

In this paper, we characterize the behavior of “big”
and “fast” data analysis frameworks, in multi-tenant, shared
settings for which computing resources (CPU and memory)
are limited, an increasingly common scenario used to increase
utilization and lower cost. We study how popular analytics
frameworks behave and interfere with each other under such
constraints. We empirically evaluate Hadoop, Spark, and Storm
multi-tenant workloads managed by Mesos. Our results show
that in constrained environments, there is significant performance
interference that manifests in failed fair sharing, performance
variability, and deadlock of resources. Show less

This paper examines the use of partial least squares regression
to predict glycemic variability in subjects with Type I Diabetes
Mellitus using measurements from continuous glucose monitoring devices
and consumer-grade activity monitoring devices. It illustrates a methodology
for generating automated predictions from current and historical
data and shows that activity monitoring can improve prediction accuracy
substantially.

This paper presents SuperContra - a Design-byContract
(DbC) framework that can ship with future PaaS
offerings to enforce lightweight contracts across different programming
systems, as-a-service. SuperContra is unique in that
developers employ a familiar, high-level language to write contracts
regardless of the programming language used to implement
the component under test. We evaluate SuperContra using widely
used, open-source software and compare its performance… Show more

This paper presents SuperContra - a Design-byContract
(DbC) framework that can ship with future PaaS
offerings to enforce lightweight contracts across different programming
systems, as-a-service. SuperContra is unique in that
developers employ a familiar, high-level language to write contracts
regardless of the programming language used to implement
the component under test. We evaluate SuperContra using widely
used, open-source software and compare its performance against
existing DbC frameworks. Our results show that SuperContra
performs on par with non-service-based DbC approaches and in
some cases similarly to code running without contracts. Show less

As scalable information technology evolves to a more cloud-like model, digital assets (code, data and software environments) increasingly require curation as web-accessible services. "Service-izing" digital assets consists of encapsulating assets in software that exposes them to web and mobile applications via well-defined yet flexible, network accessible, application programming interfaces (APIs). In this paper, we postulate that recent advances in cloud computing make cloud platforms as-a-… Show more

As scalable information technology evolves to a more cloud-like model, digital assets (code, data and software environments) increasingly require curation as web-accessible services. "Service-izing" digital assets consists of encapsulating assets in software that exposes them to web and mobile applications via well-defined yet flexible, network accessible, application programming interfaces (APIs). In this paper, we postulate that recent advances in cloud computing make cloud platforms as-a- service (PaaS) ideal for deployment, lifecycle management, and policy-based control – i.e. API governance – for extant and future digital assets. Toward this end, we overview API governance as a PaaS technology and outline some early results generated by our investigation of a prototype we are developing, called EAGER, for implementing API governance at scale. Show less

As scalable information technology evolves to a
more cloud-like model, digital assets (code, data and software
environments) that increasingly form the basis of research and
education require curation as web-accessible services. “Serviceizing”
digital assets consists of encapsulating assets in software
that exposes them to web and mobile applications via welldefined,
network accessible, application programming interfaces
(APIs). The stability, maintenance, and lifecycle of… Show more

As scalable information technology evolves to a
more cloud-like model, digital assets (code, data and software
environments) that increasingly form the basis of research and
education require curation as web-accessible services. “Serviceizing”
digital assets consists of encapsulating assets in software
that exposes them to web and mobile applications via welldefined,
network accessible, application programming interfaces
(APIs). The stability, maintenance, and lifecycle of these APIs
is critical to the utility of the digital assets they serve. Our
work focuses on the development methodologies and technologies
for API governance – policy, implementation, and deployment
functions for IT management of APIs at scale. This paper
presents our view of API governance in a technology landscape
that is trending towards reliance on web services. It also outlines
some early results generated by our investigation of a prototype
we are developing for implementing API governance at scale. Show less

In this work, we propose a novel architecture that
can be used to optimize the usage of the scarce bandwidth in
rural areas of the developing world. The architecture consists of
a proxy server that time-delays file uploads to other time periods
that the networks is underutilized and a web-server that stores
the files and manages their delayed upload.

With the low installation and maintenance cost of
IEEE 802.11-based equipment, dense Wi-Fi deployments are a
reality, especially in today’s urban areas. This vast number of
WLANs can be exploited to achieve low-cost ubiquitous wireless
Internet access, which is also demostrated by the emergence of
community-based wireless access schemes. In our prior work
we have developed a reciprocity-based peer-to-peer architecture
for Wi-Fi sharing, where peers provide free Wi-Fi access… Show more

With the low installation and maintenance cost of
IEEE 802.11-based equipment, dense Wi-Fi deployments are a
reality, especially in today’s urban areas. This vast number of
WLANs can be exploited to achieve low-cost ubiquitous wireless
Internet access, which is also demostrated by the emergence of
community-based wireless access schemes. In our prior work
we have developed a reciprocity-based peer-to-peer architecture
for Wi-Fi sharing, where peers provide free Wi-Fi access to
others in order to enjoy the same benefit when they are away
from their own Wi-Fi network. Our system tries to match peer
consumption with contribution and we have shown it to work
well for city-scale Wi-Fi sharing communities. However, when
attempting to roam outside the city boundaries, the statistics are
such that there is typically a lack of consumption-contribution
information between consuming and providing members, which
hinders the system’s scalability. In this work, we extend our
architecture with global-scale roaming capabilities by relaxing
the requirement for full decentralization. In particular, we
exploit special trusted super-peers which act as representatives
of different Wi-Fi sharing communities (e.g., communities of
different geographical regions) and which mediate transactions
when there is insufficient information about peer contribution
history. Extensive simulations show that this super-peer-assisted
approach can significantly enhance the system’s performance in
terms of roaming coverage. Show less