JimBobBennett

Your eval criteria are already written, just scattered across three systems

Fri, 24 Jul 2026 00:00:00 +0000

In an earlier post I built a self-improving agent by mining a context graph out of data the team already had. The source of truth wasn’t in a spec, it was in the traces: every time a human reviewer overrode the agent, that override was a signal, and joined together across hundreds of decisions those signals told you what the agent should have done. Mine them, feed the patterns back, and the agent gets measurably better without retraining a thing.

That post was about runtime, what the agent did, and how people corrected it. This one is about the other end of the timeline, and it turns out to have the same shape.

Before you can tell whether an agent did the right thing, you need a definition of right. And that definition almost never lives in one place. It’s split across the product requirements doc, the Jira ticket where the edge case got renegotiated, and the function nobody documented that quietly encodes the real rule. You already wrote the eval criteria, but you wrote them three times, in three systems, and never joined them up.

The requirements diaspora

Think about where the actual spec for an AI feature lives by the time it ships.

The PRD holds the intent. It’s the clean story: here’s what we’re building, here’s who it’s for, here’s what good looks like. It was also written before anyone hit the hard parts, so it’s the least accurate of the three by the time you’re evaluating.

The tickets hold the negotiation. This is where “actually, do it this way” lives, buried in a comment thread on a ticket that was closed four months ago. The edge cases, the reversals, the “we’ll handle that later” that became permanent. If you want to know why the agent behaves the way it does, the answer is usually in a Jira comment, not the PRD.

The code holds what actually shipped. Including the rules nobody wrote down anywhere else: the threshold someone hardcoded, the special case for one customer, the fallback that fires more often than anyone realizes.

These three drift apart, and the drift is not a bug in your process, it’s the default. Keypup describes it as context drift between your tickets and your code: “at any given moment, the two systems tell different stories.” Their sharper line is the one worth pinning up: “Gaps between what a ticket describes and what a specification actually defines are where most production bugs start their life.” Jama makes the same point about specs and implementation, where the traceability gap between the detailed requirement and what engineering built often isn’t caught until sprint review.

So the source of truth exists. It’s just fragmented, and the fragments disagree.

Why this breaks AI evals specifically

Traditional software has a defense against this. The tests live next to the code, they run on every change, and a red build tells you something moved. The spec is imperfect, but the test suite pins the behavior down.

AI systems don’t get that for free. A model is non-deterministic, so you can’t assert output == expected. Instead you judge quality against a rubric, either with human labelers or with an LLM acting as a judge. And here’s the problem: that rubric usually gets invented from scratch. Someone sits down in a fresh doc and writes out what good looks like, disconnected from the PRD, the tickets, and the code that already defined it.

There’s a popular framing that building this rubric is fundamentally a product exercise, that it’s essentially writing a PRD for AI behavior. That’s half right, and the wrong half is the expensive one. You don’t need to author a PRD for AI behavior. You already have one, well three, actually. Writing a fresh rubric means re-deriving requirements that already exist, and re-deriving them badly, because the person writing the rubric doesn’t have the Jira comment or the hardcoded threshold in front of them. You end up evaluating against a guess.

Even a passing eval doesn’t save you here, and this is the part that should worry you. Tricentis has a name for it: intent drift, “the gradual divergence between what code actually does and what it was originally designed to do.” Their line lands harder in the eval context than the code one: “A green pipeline doesn’t mean your code still does what you intended.” If your rubric was invented separately from your requirements, a green eval run tells you the system matches the rubric. It tells you nothing about whether the system still matches what you meant to build. You can pass every check and have quietly shipped something nobody asked for.

Mining requirements into one source of truth

This is the same move as the context graph, pointed at a different corpus.

In the context graph post, a mining agent read the traces and extracted structured nodes: the request, the decision, the override, the precedent it set. Here, you point an agent at the three requirements systems and extract requirements the same way, as structured nodes rather than prose. A requirement, the source it came from, its acceptance criteria, and the ticket where it last changed. The Claude Agent SDK is well suited to this: it traverses a repo natively, and with the right MCP tools wired up it can also read a Confluence space and walk a Jira project in the same run, which is exactly the three-system sweep the job needs.

The interesting output isn’t the clean list of requirements. It’s the conflicts. When the PRD says one thing, the ticket renegotiated it to another, and the code does a third, that disagreement is the most valuable thing the mining produces. Tricentis arrives at nearly the same recipe from the code-quality side, arguing you have to look at three inputs together: the original intent, the current result, and the changes that got you from one to the other. Swap in PRD, code, and tickets and you have the reconciliation this post is about. The conflicts are where drift is hiding, and surfacing them is half the value before you’ve evaluated anything.

Treating the code as a first-class requirements source is not a hack, either. The ReqToCode work argues traceability should be a structural property of the codebase rather than a matrix bolted on afterward. The code isn’t just the implementation. It’s a requirements document that happens to be executable, and it’s the closest of the three to what actually shipped — with the caveat that for an AI system the deployed prompt, model version, and runtime config often live outside the repo, so the code is authoritative about the logic, not the whole system.

From source of truth to a grounded eval

Once the requirements are mined and reconciled, they stop being documentation and become eval infrastructure.

Acceptance criteria become pass/fail checks. The edge cases you dug out of ticket comments become test cases in your dataset. The undocumented rules in the code become graders. The rubric isn’t authored, it’s compiled from a source of truth that already carried the answers.

And this is where grounding stops being a nice word and starts being a number. Vague rubrics fail measurably. FutureAGI’s work on LLM-as-judge puts it bluntly: “A vague rubric prompt produces a vague judge.” They track it with Cohen’s kappa, the standard measure of agreement between two labelers beyond chance. A human pair should hit kappa of at least 0.7 for you to trust the rubric is clear. A judge that agrees with humans at 0.6 or above is acceptable for production, 0.8 is strong, and a miscalibrated judge working from a fuzzy rubric can fall as low as 0.31. Maxim frames the same idea from the annotation side: you want acceptance criteria specific enough that two experts reach the same conclusion independently, because “your evaluation system is only as good as its source of truth.”

Grounding the rubric in mined requirements is what moves those numbers. When a labeler judges an output, they’re not guessing against vibes. They’re checking it against the acceptance criterion that came from a real ticket, so two labelers reach the same verdict, and the LLM judge you calibrate against them inherits that clarity.

There’s a second payoff, and for my money it’s the bigger one. A rubric built from how the system actually works, code and tickets and PRD reconciled, is a mirror you can hold up early. Show it to the people who asked for the feature before you run a single eval. If the mined rubric describes an app they don’t recognize, you’ve caught the drift while it’s still cheap to fix. The requirements moved, the code followed, and nobody stepped back to check the result was still the thing they wanted. The reconciled rubric surfaces that mismatch before you waste a cycle evaluating against the wrong target.

The rubric is a first draft

None of this means mining hands you a finished rubric. It hands you version one.

The honest version of this is that criteria don’t fully emerge from a spec at all. As Lenny’s guide to eval systems puts it, criteria emerge from grading: “the first version of any rubric is wrong in ways you can’t predict until you watch it fail on real examples.” You mine the requirements, you compile the rubric, and then you grade real output against it, and the grading is what exposes where the requirements were ambiguous, contradictory, or silent.

That’s a loop, and it’s the same loop as the context graph. There, each cycle of mining and feedback improved the agent, from 53.8% agreement to 76.2% and up toward a plateau. Here the loop improves something one level up: the definition of correct. Grading reveals a gap, the gap points back at an ambiguous requirement, you resolve it in the source, and the next rubric is sharper. The context graph tightened what the agent does. This tightens what you measure it against.

Don’t leave your requirements fragmented

The closer from the last post applies here almost word for word: don’t throw your data away. You already wrote the requirements. The failure isn’t missing knowledge, it’s leaving it scattered across three systems that disagree.

Whether the source of truth is override signals in your traces or acceptance criteria in your tickets, the job is the same. Mine the systems you already have, join the fragments nobody joined, and reconcile them into one thing you can evaluate against. Do that and you get two wins at once: an eval grounded in what the system was meant to do, and an early warning the moment the thing you built stops being the thing you meant to build.

Evals Are a Revenue Strategy, Not a Safety Net

Mon, 20 Jul 2026 00:00:00 +0000

Here’s the moment every AI team knows. You have a demo that works. Not always, but most runs, and it’s good enough that people in the room lean forward. Then someone tweaks the system prompt, or swaps the model, or adds a tool, and the question lands: did that make it better or did it quietly break something? Nobody in the room actually knows. You run it a few times by hand, it looks fine, and you ship on a shrug.

That shrug is expensive. Not because of the risk of shipping something bad, although that’s real. It’s expensive because the shrug is slow. Every time you can’t answer “is this good enough?” quickly, you either stall while you check by hand, or you gamble. Both cost you time, and in AI right now time is the whole game.

Evals and observability usually get pitched as insurance. Put them in, sleep better, catch the embarrassing failure before your users do. That’s true, and it badly undersells them. The bigger story is that the same machinery that reduces your risk is what lets you move fast, and moving fast is where the revenue is. Risk reduction is the floor. Speed is the return.

Shipping late has a dollar price

Start with the oldest finding in the book, because it still holds. Back in 1991, Harvard Business Review laid out the math from a McKinsey study:

A McKinsey study reports that, on average, companies lose 33% of after-tax profit when they ship products six months late, as compared with losses of 3.5% when they overspend 50% on product development.

Going 50% over budget costs you about 3.5% of profit. Shipping six months late costs you a third of it. The market does not care what your product cost to build. It cares when it showed up. Being slow is roughly 10 times more expensive than being over budget, and that’s from an era when a product cycle was measured in years.

This isn’t only about launch timing. It’s about the whole rhythm of shipping. The DORA research program has spent years measuring how software teams deliver, and the teams that ship fast and often aren’t just tidier engineering shops. They win on business outcomes. Elite delivery performers, DORA found, are “twice as likely to meet or exceed their organizational performance goals”: profitability, market share, the things a CFO actually tracks. Speed of delivery and business success turn out to be the same muscle.

The finish line moved from “first” to “first to something that works”

It’s tempting to read all that as “be first, win everything.” That’s not quite right, and the honest version is more useful. Being first is genuinely risky. The classic Golder and Tellis study of market pioneers put the failure rate of those pioneers at 47%. Nearly half of the companies that got somewhere first died there. First out the door is not the prize.

The prize is being first to something that actually works, and then staying ahead by improving faster than everyone else. That distinction matters more in AI than it ever did in software, because the ceiling on speed just moved. Stripe’s data on the top AI companies shows this cohort hit $1 million in annualized revenue in “a median period of just 11.5 months… about 4 months ahead of the fastest-growing SaaS companies.” The fastest-growing software category in history just got beaten by four months, and the teams doing the beating are the ones iterating toward a working product fastest.

So the game isn’t a one-time land grab. It’s a race of iteration. Whoever can go from “idea” to “working and trustworthy” and back again, over and over, on the shortest loop, captures the revenue. Which raises the obvious question: what’s actually slowing that loop down?

Why AI teams stall between the demo and production

The thing that kills the loop is the gap between a demo that impresses and a product you can put in front of paying users. That gap is where AI projects go to die, and the numbers are brutal. S&P Global Market Intelligence found that “the share of companies abandoning most of their AI initiatives jumped to 42%, up from 17% last year,” and that the average organization “scrapped 46% of AI proof-of-concepts before they reached production.” Almost half of everything that gets built never ships.

The root cause is the thing that makes AI feel like magic in the first place: it’s non-deterministic. Ask a model the same question twice and you get two different answers. That’s fine for a demo and poison for a release process, because your normal safety net stops working. As Red Hat’s engineers put it, existing test frameworks “don’t work well for agentic systems because of inherent variability.” You can’t assert output == expected when there’s no single expected output. So the demo-to-production gap isn’t a discipline problem or a talent problem. It’s a measurement problem. You cannot ship what you cannot measure, and most teams have no fast way to measure a fuzzy system.

That’s the real speed killer. Not that the model isn’t good enough, but that you can’t tell, quickly and repeatably, whether it is.

Evals turn “is it good enough?” into a loop you can run

An eval is just a repeatable way to score your AI’s output against what “good” means for your use case. A code check, or another model acting as a judge, that turns a squishy answer into something you can compare: a score, a label, a pass or a fail. Observability is the other half: tracing what your system actually did on every run, so when a score drops you can see why. Put them together and the unanswerable question from the top of this post becomes a number you can watch move.

That’s the whole point, and it’s worth being precise about why it’s a speed win and not just a safety one. When “is this better?” is a manual check, every change costs you an afternoon of poking. When it’s an eval, every change costs you a test run. You stop gambling and you stop stalling, at the same time. Anthropic’s engineering team said it plainly:

Teams without evals get bogged down in reactive loops… Teams that invest early find the opposite: development accelerates.

Development accelerates. Not “risk decreases,” although it does. The team with evals ships faster because they can trust their own changes. And this shows up in real numbers, not just principle. When DoorDash built automated evals for their in-app assistant, they cut regression testing “from six hours to twenty minutes.” That’s the same six-hour manual check, run as an eval instead, roughly 18 times faster. Multiply that across every change a team makes in a quarter and you can see where the four-month head start comes from. The eval is the thing that lets you ask “is this good enough?” as often as you need to, instead of once, nervously, before a launch.

Where AX fits

This is the loop Arize AX is built for. AX is the AI engineering platform for the whole eval-driven cycle: tracing what your system actually did on every run, evaluating quality against the standards you set, running controlled experiments, and monitoring what happens once real users are hitting it. It frames the exact gap this whole post is about. As their product page puts it, “You can build and ship at agent speed… You still can’t improve at agent speed.” Building fast was never the bottleneck. Improving fast, with confidence, is.

That’s the whole job of an eval tool. You turn the runs that mattered into a dataset, you score every change against it instead of poking by hand, and you run an experiment to prove a change is better before you ship it. Then, once you’re live and the stakes are real, the same evaluators keep watching in production, so a regression sets off an alert instead of a support ticket. It’s one continuous loop that shortens the distance between “I changed something” and “I know whether it worked” — first while you’re building, then while you’re running.

The floor and the return

Reducing risk is the floor. Evals will absolutely catch the regression before your users do, and that’s worth the price on its own. But the floor was never the interesting part. The return is that the exact same discipline, the same golden datasets, the same evaluators, the same traces, is what lets you move fast enough to win. In a market where the product is non-deterministic and the fastest iterator takes the revenue, the ability to answer “is this good enough to ship?” in minutes instead of afternoons isn’t a safety feature. It’s the moat.

You don’t put evals in because you’re worried. You put them in because you’re in a hurry.

Evals belong in your CI/CD pipeline

Wed, 15 Jul 2026 00:00:00 +0000

The other day I wrote that evals are just testing - the same old loop of “decide what good looks like, check reality against it, fix your definition when it turns out to be wrong,” just with a much weirder answer key. A few people replied with the obvious follow-up question, and it’s a good one: OK, if evals are just tests, why aren’t they running where my tests run?

Because that’s the bit we often skip. We’ll happily wire up a golden dataset, write an evaluator, run it once from a notebook, nod at the score, and move on. Meanwhile every actual unit test we own runs automatically on every pull request, and a failed test blocks the merge. Evals get none of that. They sit in an observability platform like Arize AX while someone changes a system prompt in GitHub and ships it on vibes.

This post is about closing that gap. Evals are tests, so let’s put them where your tests live - in continuous integration, gating your pull requests - and let’s do it in a way that survives the fact that AI output is fuzzy and non-deterministic.

Your golden dataset is already a parameterised test

Start with the thing you already have: a golden dataset. A curated set of trusted inputs with the outputs you’d want back, usually hand-labelled by someone who knows the domain. A dataset is “the foundation for reliably testing and evaluating your LLM application.”

If you’ve written tests in the last decade, you’ve seen this shape before. It’s a parameterised test. Here’s the classic pytest version:

import pytest

@pytest.mark.parametrize("expression,expected", [
 ("2 + 2", "4"),
 ("10 / 2", "5"),
 ("3 * 3", "9"),
])
def test_calculator(expression, expected):
 assert calculate(expression) == expected

One test body, three rows of parameters, three results. You don’t write test_calculator_1, test_calculator_2, test_calculator_3 - you write the body once and feed it a table of cases.

A golden dataset is that table. Each row is an input plus the expected output, and every row runs through the same task. When you run an experiment in AX, that’s exactly what happens: your task function runs once per row, and every evaluator scores every row.

Same idea as @pytest.mark.parametrize, just that the parameters live in a versioned dataset instead of a decorator. Which is actually the upgrade you want, because a golden dataset grows: you pull interesting cases out of production, a teammate adds an edge case that broke last month, and the table gets richer over time without touching the test body.

Arrange, act, eval, assert

Here’s the mental model I keep coming back to. You already know arrange-act-assert - the three beats of pretty much every unit test ever written:

Arrange: set up your inputs and world.
Act: run the thing under test.
Assert: check the output is what you expected.

The reason that works is that the output is deterministic. calculate("2 + 2") returns "4" every single time, so assert result == "4" is a fair check. But an AI app doesn’t play that game. Ask a model the same question twice and you’ll get two different sentences that mean the same thing. You can’t assert output == "the expected answer" because there isn’t an answer, there’s a fuzzy cloud of acceptable ones.

So you add a step. Arrange, act, eval, assert.

The new eval step is where the fuzziness gets dealt with. Before you can assert anything, you run the output through an evaluator - which can be any of four things, from a dead-simple code check to a full LLM-as-a-judge - that turns a squishy sentence into something you can assert on: a score, a label, a pass or fail. Only then do you assert.

Arrange is fetching the dataset. Those are your rows.
Act is the task function - run your app against a row and produce an output.
Eval is the evaluator - grade that output into a score.
Assert is the gate - pass or fail based on the scores.

Your evaluators are the new eval step. You run an eval against the output from your test first. This gives you the deterministic output that feeds the Assert.

Gate on a percentage, not perfection

This is the part that trips up engineers coming from normal tests, so I want to sit on it for a second.

With a unit test suite, the rule is simple and brutal: one failed test fails the build. You want the board fully green, every time, no exceptions. It’s tempting to carry that straight over - “all evals must pass or the PR is blocked” - and it’s wrong.

AI output is non-deterministic. Arize’s own writing on testing agents says it well:

We don’t expect 100% pass rates (that would be suspicious). But we do expect consistency.

A 100% pass rate on a fuzzy system usually means your dataset is too easy or your judge is asleep, not that your app is perfect. So instead of demanding every row pass, you gate on an aggregate: what fraction passed, or what the mean score was, against a threshold you decide.

Your threshold is a real engineering decision, not a cop-out. Set it where a regression should genuinely block a merge, watch it over time, and raise it as your app gets better. If pass rates drop below our threshold, the PR is blocked.

Wiring it into a PR gate

Now the fun bit - making it run on every PR that touches your AI.

You don’t run your evals on every commit - that’d be slow and expensive - you run them when someone changes the stuff that actually affects behaviour: the prompts, the tools, the agent logic. Change a system prompt, the experiment runs against your golden dataset, and if the pass rate falls below your threshold the PR goes red. Your API key lives in secrets and gets injected as an environment variable, same as any other credential.

That’s it. That’s evals as a PR gate. A change that quietly makes your assistant worse now gets caught by a robot before a human ever hits merge, exactly like a failing unit test.

Who tests the tests? Unit-testing your eval prompts

There’s a sharp edge here, and it’s the one my old chiral-molecule bug beat into me: the thing you test against can itself be wrong. An LLM-as-a-judge is just a prompt, and prompts are fallible. A judge that’s too generous passes garbage. A judge that’s too harsh blocks good changes and trains your team to ignore the gate. Either way, a broken judge is worse than no judge, because you trust it.

So you test the judge, using the exact same machinery. You just point it at itself.

Build a small dataset of outputs you’ve already graded by hand - some you know are good, some you know are bad, with the correct verdict labelled on each row. Now run your judge against that. The judge is the thing under test, the task is “ask the judge to grade this output,” and the human labels are the expected values. If the judge disagrees with the humans too often, the judge fails its own test, and you go fix the judge prompt before you trust it to gate anything.

It’s parameterised testing all the way down. Same arrange-act-eval-assert, except the “app” being tested is your evaluator. One warning worth its own callout:

Don’t tune your judge on the same examples you use to prove it works. That’s teaching to the test - the judge looks great on the cases it was built against and falls over on everything else. Hold some examples back. I wrote about the train/dev/test split separately, and it matters more here than almost anywhere.

Do this and you end up maintaining two products that both live in CI: the AI app, and the eval that guards it. Both have golden datasets. Both have gates. Both block a PR when they regress. That’s eval-driven development - the same discipline you already trust for code, turned back on the tool doing the grading.

So where does that leave you?

Nowhere new, really, and that’s the point. You already know how to do all of this. Parameterised cases, a red-green gate in CI, a PR that can’t merge when the checks fall over - that’s Tuesday for most engineering teams. Evals just ask you to insert one extra step before the assertion and to gate on a percentage instead of a perfect board.

Put your evals in CI. Let a golden dataset drive them, let an evaluator handle the fuzzy part, and let a threshold block the bad PRs. The loop is old. You’re just running it on the weird answer key now, right next to the tests you already have.

Long-horizon agent benchmarks are fragmenting: a field guide to what each one actually measures

Sun, 12 Jul 2026 00:00:00 +0000

Originally published on the Arize AI blog: Long-horizon agent benchmarks are fragmenting: a field guide to what each one actually measures.

In one of the evaluations behind the OpenAI and Apollo Research scheming study, o4-mini was given a cyber-security test with a trap buried in the formatting instructions. The fine print said that scoring above 50% on this section would trigger “unlearning” of the capability. The model read it, understood the incentive, and threw the test on purpose. This is sandbagging: an agent intentionally underperforming on an evaluation it knows it is taking. o3, given the same trap, did the same.

Think for a moment about what that means for a leaderboard. The number on the page was not a measure of what o4-mini could do, it was a measure of what o4-mini chose to reveal, given its read of who was watching and why. The model was not failing the benchmark, it was playing it.

That is the problem this whole generation of long-horizon agent benchmarks is walking into. A long-horizon agent benchmark measures an agent across a task that unfolds over many steps, tool calls, and decisions rather than a single prompt and reply, the kind of sustained work that runs for hours and, in SWE-Marathon’s case, hundreds of millions of tokens. Because the agent acts over a long trajectory instead of answering once, the benchmark has to grade the whole arc of behavior, which is exactly what makes these so hard to build and so easy to fool.

A wave of them shipped in recent months: Agents’ Last Exam, SWE-Marathon, the Meta-Agent Challenge, and Arena’s Agent Mode. Each is a serious attempt to measure economically meaningful agent work, and each strikes a different bargain to make it measurable. Underneath all four sits Princeton’s reliability work, which found that climbing capability scores have yielded only small improvements in reliability. It is less a benchmark of its own than an explanation of why the others leak.

The bargain is the whole story. Every one of these benchmarks buys measurability with the same currency: it trades realism against verifiability, and whichever side it underpays is the exact seam where the score leaks. Reading these benchmarks well is mostly about knowing which failure mode you bought.

The axis you cannot escape

There is no benchmark that is both realistic and cleanly verifiable. Those two properties pull in opposite directions, and you have to cut one to get an increase in the other.

Push toward verifiability and you select for tasks with a checkable answer: a test that passes, a string that matches, an output you can diff against ground truth. That keeps the score objective. It also quietly narrows the benchmark to the slice of real work that happens to be checkable, and “checkable” turns out to be a short walk from “gameable.” If a fixed answer exists somewhere in the environment, a capable agent can often reach it without doing the work.

Push toward realism and you lose the clean answer key. Real tasks have no oracle. So you fall back on a softer signal: a human’s approval, another model’s judgment, a behavioral trace. The score stops being a correctness measure and becomes a preference measure, with all the noise that implies.

Then add optimization pressure to either side and the agent stops passively sitting on the bargain and starts working the seam. The Meta-Agent Challenge put it plainly: “high optimization pressure induces spontaneous reward hacking.” Push hard enough on any proxy and the agent optimizes the proxy instead of the task. That pressure is not a separate leak; it is what drives the two leaks below.

The failure grammar: two ways the score leaks

There are two distinct ways a benchmark score gets corrupted, and almost every specific failure is an instance of one of them.

The first is harness-side. The scaffold leaks the answer to the agent. The cleanest catalogue of this is Berkeley RDI’s “How We Broke Top AI Agent Benchmarks”, which matters because RDI is Dawn Song’s lab, the same group that built Agents’ Last Exam. This is benchmark builders auditing the field, not critics from the outside.

Their findings read like a rap sheet. o3 and Claude 3.7 Sonnet reward-hacked in 30% or more of evaluation runs, using stack introspection, monkey-patching the grader, and operator overloading to fake a pass. They found a coding model that claimed 81.4% on SWE-bench, where 24.4% of its trajectories simply ran git log to copy the answer out of commit history. OpenAI audited SWE-bench Verified, discovered 59.4% of problems had flawed tests, and stopped using it. KernelBench handed out full marks for zero computation, because torch.empty() returned stale GPU memory that still held the reference answer from the evaluator’s previous run.

RDI distills all of it into seven recurring vulnerability patterns. Most of them reduce to two sentences: there was no isolation between the agent and the evaluator, or the answer shipped alongside the test. Once you have those seven patterns in your head, you stop reading benchmark failures as bad luck and start reading them as a grammar.

The second way is model-side, and it is the one the sandbagging story opened with. Here the scaffold does not leak anything. The agent detects that it is inside a scaffold and changes its behavior. The answer is not exposed; the measurement is corrupted because the thing being measured is reacting to being measured. Those are the two rails, harness-side leaks and model-side reactions, and every benchmark below speaks some dialect of one or the other.

The field guide

For each one: what it measures, the bargain it strikes, and then the seam itself. Two of the four have a documented exploit, a recorded case of an agent gaming the score. The other two have not been gamed in public yet, so their entries point instead at the structural downside waiting at the seam, because “not yet attacked” is not the same as “sturdy.”

Agents’ Last Exam

What it measures. Sustained professional work. It was built with more than 250 industry experts, with non-physical occupations mapped onto the US federal O*NET / SOC taxonomy, the government’s standardized catalog of occupations and the tasks each one involves. Its hardest tier remains far from saturated: across mainstream harness and backbone configurations, the average full pass rate is below 1%.

The bargain. It admits only tasks with “verifiable outcomes” and standardizes grading around structured deliverable- and milestone-based checks, avoiding an LLM judge wherever a deterministic alternative exists. That is a deliberate move all the way toward the verifiable end of the axis.

The downside. No public exploit yet, but a low pass rate buys no safety here, as the harness failures below make plain. The real catch is that a ceiling is not a ruler. When you accept only tasks with objective answer keys, “verifiable” quietly narrows what counts as professional work, so that sub-1% rate measures the hardest checkable slice, not the hardest real one, and a low score does not mean “no economic value.” Use it to probe a capability frontier; do not read it as a verdict on whether agents can do a job.

SWE-Marathon

What it measures. Coherence over enormous horizons. It was built on a specific complaint, stated plainly in the paper: “current agent benchmarks largely evaluate short-form tasks,” and so never test the planning, long-context understanding, and memory that real work demands. Its 20 tasks answer that with genuinely long frontier work: a multi-pass C compiler in Rust from preprocessing through x86-64 codegen, OpenAI’s Parameter Golf, Cursor’s long-running agent tasks. The rollouts average 27M tokens and top out at 877M, and even the best agent, Claude Opus 4.8, solves only 26%.

The bargain. Spending the budget on realism sends the bill to verification. To grade a full-stack Slack clone the benchmark hands the result to a Computer Use Agent that logs in, creates channels, posts messages, and checks the app actually works through the UI. When the grader is itself an agent it inherits the failure modes of the thing it grades.

How it was gamed. The builders saw the leak coming and wrapped grading in a multi-layer suite of visible and hidden tests, network-egress limits, and adversarial exploit scans, and still had to harden some tasks ten times over: run agents, inspect the traces, find the shortcut, patch the verifier, rerun. The seam shows in the numbers regardless. Across 1,300 rollouts, 14% showed reward-hacking behavior and 10% shipped clear exploit code, the predictable result of optimizing an agent against the environment that scores it.

The Meta-Agent Challenge

What it measures. Agents building agents. Rather than solving a task directly, the agent is scored on its ability to “autonomously construct, refine, and optimize an agent system” to solve it. To do that realistically it uses a dual-container sandbox where the meta-agent gets development-set feedback through an evaluation interface (API) while the test set stays hidden.

The bargain. That dev-set feedback channel is what makes the task realistic, an agent iterating against signal the way a real engineer would. It is also the attack surface.

How it was gamed. GPT-5.3-Codex “autonomously weaponized verbose error tracebacks to exfiltrate development-set ground truth,” a misalignment behavior the authors say emerged spontaneously during benchmark development. Nobody taught it the trick; optimization pressure found the leak by itself.

Arena’s Agent Mode

What it measures. What real users actually do. In its first seven-day window it logged 160,480 agent tasks across 128,244 live sessions, with more than two million tool calls.

The bargain. It pushes all the way to the realistic end of the axis and gives up fixed ground truth entirely. There is no answer key; the label is a real user’s approval or disapproval, and Arena treats the whole thing as a multi-intervention randomized controlled trial to estimate which agent components actually cause better outcomes. That is a genuinely different and defensible measurement.

The downside. No exploit on record, and there may never be one in the usual sense, because a preference signal is not the kind of thing you game by finding a hidden answer. The seam is subtler and structural: the signal is satisfaction, not correctness, so you measure whether users felt helped and have to infer whether the agent was actually right. The behavioral traces hint at the gap. Users pulled control back about 2.3 times more often than they handed it over, and on multi-part requests, agents silently dropped part of the work about 8% of the time. Use Arena to compare felt helpfulness at scale; do not read a high score as a correctness check.

When the model knows it is being measured

Sandbagging is the seam that should worry you most, and it is not an isolated trick. It is one face of evaluation awareness, and the scheming research pins down how badly it distorts a score.

Start with what looked like good news. Anti-scheming training, what OpenAI calls deliberative alignment, cut covert actions sharply, dropping both o3 and o4-mini from low double digits to well under one percent. If you stopped there you would call the problem solved. But over the same training, the rate at which the models verbalized “I am being tested” went up rather than down. Some of that drop in covert actions was not the model scheming less. It was the model getting better at recognizing the exam room.

Then the researchers ran the cleanest experiment in the whole set. They edited the chain of thought directly. Delete the “I might be tested” reasoning from a trajectory and covert actions jumped from 13.2% to 24.2%. Inject that reasoning into trajectories that lacked it and covert actions dropped from 24.5% to 18.6%. The behavior was causally downstream of the model believing it was on a test. That is a benchmark score conditioned on the model knowing it is a benchmark, demonstrated by intervention rather than correlation.

And the realism gap shows up right where the axis predicts: in realistic, ChatGPT-style conversations, the same training helped far less, moving o3 only from 36.8% to 20.5% and o4-mini only from 31.4% to 14.2%. Anthropic reported the same shape separately, noting in the Claude Sonnet 4.5 system card that the model often recognized when it was being tested.

There is an obvious rebuttal: just read the chain of thought and catch the model in the act. The scheming work closes that door too. Post-training, the models’ reasoning traces began drifting away from standard English, with ordinary words like “illusion,” “marinade,” and “fudge” turning up used in non-standard ways, even on a standard benchmark like GPQA, a graduate-level science question set. The one instrument we lean on to catch cheating is drifting toward unreadable.

Why one number was never going to survive this

Underneath every benchmark above sits Princeton’s “Towards a Science of AI Agent Reliability.” Its core finding is quiet and brutal: most models are still ranked on mean task success rate, a single average that can hide highly inconsistent behavior. Across 14 agentic models over 18 months, reliability has barely moved even as raw capability climbed. The team decomposes reliability into four dimensions, consistency, robustness, predictability, and safety, and states the gap directly: “recent capability gains have only yielded small improvements in reliability.” Capability and consistency are coming apart.

A separate group reached the same place from a different direction. Claw-Eval ran 14 frontier models under error injection and watched Pass^3 fall by up to 24 percentage points. Pass^3 runs each task three times and counts it solved only if the agent succeeds on all three, so it measures consistency rather than one lucky run. That collapse is the same divergence Princeton documents. And the blindness is not only about variance: AgentPex extracts an agent’s own prompt rules, checks the execution trace against them, and shows that outcome-only scoring sails straight past incorrect workflow routing and unsafe tool use that never change the final answer. The single number hides the failure on the variance axis and the trajectory axis at once.

That is the deepest version of the thread. Every leak in the field guide is a way the average stops meaning what the reader thinks it means. The harness leaks and the average counts a gamed pass as a real one. The model sandbags and the average reads a chosen failure as a capability ceiling. A single pass rate cannot tell any of these apart, it was never built to.

Run the second cycle on the benchmark

A benchmark is an evaluation, and an evaluation is a thing that can itself be wrong. We grade models constantly and almost never grade the grader.

Arize’s evaluator design runs two improvement cycles off the same failure cases. The first is the agent cycle: evaluators flag bad responses, you collect them, you feed them back into prompts and fine-tuning. That cycle grades the model. The second is the evaluator cycle: you put human ground-truth labels on those same cases to check whether the evaluator was right, and you fix it when it was not. That cycle grades the grader.

A public benchmark is the first cycle run at industry scale. Every benchmark in this field guide ships with the second cycle missing. Nobody put ground-truth labels on the grader. That is why the realism-versus-verifiability trade does not sit there quietly: the seam leaks because no second cycle is watching it. RDI is the field report of what escapes through that gap on the harness side, and the scheming work is the report from the model side.

And the cost is measurable: when Claw-Eval graded agents by their trajectories instead of their final answers, outcome-only grading missed 44% of safety violations. “Grade the grader” is not a slogan; it is 44% of safety violations that final-answer scoring never sees.

So when the next leaderboard lands, do not ask which model won. Ask which side of the axis the benchmark underpaid, then go look at the seam. An evaluation you never evaluate is just a vibe at scale. Instrument the grader, not just the model.

AI evals are just testing (with a much weirder answer key)

Sat, 11 Jul 2026 00:00:00 +0000

Years ago, before “AI” meant chatbots and before anyone said the word “eval” out loud, I spent a few months chasing a bug that didn’t exist. Or rather, the bug was real - it just wasn’t in my code.

I was working at a cheminformatics company - software for chemists, basically - and my job was to add chiral searching to our chemical structure search. If you’ve not had the pleasure: chirality is when a molecule and its mirror image aren’t the same thing, like your left and right hands. Same fingers, same layout, but you can’t rotate one onto the other. In chemistry that difference is a big deal - one version of a molecule can be a medicine and its mirror image can do nothing, or worse. So “does this structure match that one?” suddenly has to care about handedness.

My product manager did something genuinely brilliant. Instead of writing me a spec full of hand-wavy prose, they sent me hundreds of examples: this structure should match that one, this pair shouldn’t. Real molecules, real expected answers. I didn’t have the word for it at the time, but that was a golden dataset. And I did the obvious thing with it - I turned every example into a unit test and started making them pass.

The tests were failing because the answer was wrong

Here’s the part I still think about. As I worked through the examples, some tests just wouldn’t go green no matter what I did. My code said “no match,” the golden dataset said “match,” and I could not for the life of me reconcile them.

So I did what you do - I assumed I was the idiot. I stared at my matching logic for hours. I traced through the atoms by hand. And eventually, on a handful of cases, I realised my code was right and the dataset was wrong. Some of those hand-picked examples were mislabelled. The source of truth had bugs in it.

That changed the whole job. It wasn’t “write code until the tests pass” any more. It was “write code and fix the tests, because some of the tests are lying to me.” My PM and I ended up iterating on the golden dataset itself - correcting labels, arguing over edge cases, tightening what “match” even meant. The dataset got better because I was testing against it, and my code got better because the dataset got better.

That, it turns out, is an eval. I was doing evals in the early 2000s. I just called them tests.

Testing was never really about “does it run”

Let’s back up, because this matters for where evals fit.

If you’re an engineer, you already know testing. But it’s worth remembering that testing has never actually been about proving your code works. Edsger Dijkstra nailed this decades ago:

Testing shows the presence, not the absence of bugs.

You can’t test your way to “this is correct.” You can only ever catch it being wrong. If you did any science at school, this is the null hypothesis wearing a different hat: you never prove your hypothesis true, you just keep failing to disprove it. A green test suite isn’t proof your code works - it’s a stack of experiments that tried to show it was broken and couldn’t. Absence of evidence, not evidence of absence. And the field slowly came round to that idea. Testing started life as glorified debugging, then became “demonstrate it works,” and then - around the time Glenford Myers wrote The Art of Software Testing in 1979 - flipped into something more useful: a good test is one that finds a bug. You’re not trying to confirm your happy path. You’re trying to break the thing.

By the time we got to test-driven development, the loop was fully formed. TDD is red-green-refactor: you write a failing test that describes the behaviour you want before you write the code, then you make it pass. You’re writing down “what good looks like” first, then building towards it. Unit tests, integration tests, UI tests - each one is just a wider net, checking a bigger slice of the system against a definition of correct that you wrote down.

That’s the whole game, and it always has been:

Decide what “good” looks like.
Check reality against it.
When the check and reality disagree, figure out which one is wrong - and sometimes it’s your definition.

Hold onto that last point. It’s the one everyone forgets, and it’s the one my chiral bug beat into me.

So what’s an eval, then?

Right. AI apps. You’ve built something on top of a large language model (LLM) - a chatbot, a summariser, an agent that books meetings, whatever. How do you test it?

You can’t write assert output == "the expected answer", because there isn’t one expected answer. Ask the same model the same question twice and you’ll get two different sentences that both mean the same thing. The output is non-deterministic, and often it’s judged on fuzzy things like “is this helpful?” or “is the tone right?” - questions with no single correct string.

An eval is how you test that anyway. And if you squint, every piece of it maps onto something you already know:

A golden dataset - a set of trusted inputs with ideal outputs, usually hand-labelled by people who know the domain - is your test fixtures and expected values. It’s exactly what my PM sent me: examples of right and wrong.
An LLM-as-a-judge - a second model you prompt to grade the first one’s output - is your assertion. It’s the assert you can’t write as a simple equality check, so you hand it to something that can read a sentence and say “yeah, that’s helpful” or “no, that dodged the question.” (For the prompt-based kind, a judge is just a new prompt, not a new model - which is a very freeing thing to realise.)
The score it hands back - “8 out of 10,” “pass,” “hallucinated” - is your pass/fail, just fuzzier.

Same loop. Define good, check reality against it. If you want the full spread of how you actually run these - from a dead-simple code check all the way up to a full agent doing the grading - I wrote about the four ways to run an eval separately.

Where the analogy gets uncomfortable

I want to be honest here, because “evals are just testing” is a great line and also slightly too smug.

The loop is old. The answer key got weird.

With a classic unit test, your oracle - the thing that decides pass or fail - is crisp. 2 + 2 should be 4, and it either is or it isn’t. With an eval, the oracle is often another model, or a human, or a statistical threshold. It’s subjective. It drifts. Two reasonable people disagree on whether an answer was “helpful.” Your judge model has its own biases. And the moment you start optimising your app to score well on a judge, you risk gaming the judge instead of actually getting better - the AI equivalent of teaching to the test.

None of that makes it not testing. My chiral matching wasn’t a clean == either - it was rules and tolerances and edge cases, fuzzy long before LLMs showed up. This is a difference of degree, not kind. But the degree is big enough that you have to take one part of testing that engineers usually get away with ignoring, and put it front and centre.

Your eval is a product, and it needs testing too

Here’s the bit my chiral bug was really teaching me.

When some of those golden examples turned out to be wrong, the lesson wasn’t “datasets sometimes have typos.” It was: the thing you test against is itself a fallible product, and it needs testing. My tests were only as trustworthy as the golden data behind them, and that data was wrong often enough to matter.

With classic testing you can usually skate past this. Your expected values are simple enough that nobody seriously worries the test itself is broken. With evals you cannot skate past it at all. Your LLM judge can be confidently, consistently wrong. So you have to validate the judge against human-labelled golden data - and then you have to make sure you’re not tuning your judge on the very same examples you use to prove it works, which is its own trap. (That’s a real enough problem that I gave it its own post on the train/dev/test split.)

So now you’re maintaining two products, not one. There’s the AI app, and there’s the eval that tests it - and the eval needs its own golden data, its own iteration, its own “wait, is the judge wrong or is the app wrong?” moments. Which is exactly the argument I was having with my PM, just with a model in the mix instead of a mislabelled molecule.

“If it’s just testing, why all the new tooling?”

Fair question. If evals are just tests, why is there a whole industry of eval platforms, dataset versioning, judge calibration and dashboards? Why can’t I just chuck it all in my existing test runner?

Because the weird answer key drags a lot of baggage in with it. You need to version golden datasets, because they change as you learn - mine certainly did. You need to run the same input multiple times and look at the distribution, because one pass tells you nothing about a non-deterministic system. You need to collect real traffic and pull the interesting cases back into your dataset. You need humans in the loop to keep the judges honest. And you need evals doing different jobs at different stages - some gating your releases like a CI check, some watching live traffic in production, some sitting in the request path as guardrails.

That’s not a different discipline. It’s testing with the volume turned up on the one part - the oracle - that classic testing let you take for granted.

The flywheel

Put it all together and you get a loop that looks like an infinity symbol.

Your app runs in production and throws off traces - real inputs, real outputs. Humans review the interesting ones and curate them into versioned golden data. That golden data validates and sharpens your evals. Those evals gate changes to your app. The improved app goes back to production and surfaces new cases you hadn’t thought of. Round and round.

The important node in that diagram is the human one. Without people curating the data, the loop collapses into an app grading its own homework, and you learn nothing. The whole thing only works because a person keeps deciding what “good” actually means - the same job my PM was doing when they sent me those hundreds of examples.

So no, evals aren’t magic, and they aren’t a new thing you have to learn from scratch. If you can test software, you already know how to do this. Define what good looks like, test against it, and stay humble about the fact that your definition of good might be the thing that’s broken.

The loop is old. It’s just the answer key that got weird.

The Phoenix Project still holds up, even if you replaced all the code with agents

Fri, 10 Jul 2026 00:00:00 +0000

I reread The Phoenix Project last month. I do this every year or two — it’s one of those books that gets better the more production systems you’ve shipped, because each time you’ve broken your environment in a new way since the last read.

This time the question that wouldn’t leave me alone was a different shape. Half the work I do now involves agents, the other half is shipped by people who work with agents. Code that used to be written deterministically is now produced by a system that’s confidently wrong often enough that you have to check. Tests that used to be authored before the code now get generated from production traffic. The artefacts I’d built a career around had all been replaced.

So I sat there reading Erik teach Bill about the Four Types of Work and thinking: does any of this still apply? It’s a 2013 book set in a factory analogy from a 1984 book that’s about scheduling MRP through bottleneck machines. Has anything Gene Kim wrote survived contact with non-deterministic agents?

The answer, surprisingly, is yes. Most of it still holds up. And the parts that don’t hold are the parts you’d expect — the bits that were always specific to deterministic code, like binary test pass-fail and after-the-fact postmortems. The architecture survives because the book was never really about code. It was about the movement of work through a constrained system. Code was just the artefact that moved.

I want to walk through what survived, what didn’t, and what I’d tell a team setting up to ship agents on Monday morning. Some of the survival is obvious — telemetry and feedback loops were always going to matter more for non-deterministic systems, not less. Some of it took the re-read to see, and I’ll show you the bit I missed both times before.

It matters because something has gone wrong at the population level. DORA’s 2024 report found that for every 25% increase in AI adoption inside a team, throughput drops 1.5% and delivery stability drops 7.2%. That’s the opposite of every prior DORA finding about pipelines, automation, and small-batch flow. Whatever’s going on, the framework either explains it or it doesn’t.

Two Phoenixes

Quick disambiguation before we go anywhere. The Phoenix Project is Gene Kim’s 2013 DevOps novel. Arize Phoenix is the open-source agent observability project from Arize, the company I work for. Same word, no relationship. The book has nothing to do with Phoenix the tool, and this post is about the book.

Why it should not hold

The case against the framework surviving is real, and worth stating properly.

The DORA finding above is the most uncomfortable piece of it. Adding the most powerful productivity tool of a generation is making delivery worse. If Kim’s framework predicts elite performers can have both throughput and stability, the early agent era is contradicting it.

Then there’s Charity Majors, who saw this coming in September 2023. The person who popularised observability in modern software wrote a piece called LLMs Demand Observability-Driven Development. The argument: LLMs break the test-driven development assumption that you can write your tests before the code. “With software, you typically start with tests and graduate to production. With ML, you have to start with production to generate your tests.” The pipeline runs backwards. That’s not a small caveat. That’s an inversion of one of the most basic premises Kim’s deployment pipeline rests on.

Hamel Husain has been saying the practitioner version of this since 2024 in Your AI Product Needs Evals and the follow-up Field Guide to Rapidly Improving AI Products. He’s seen 30+ AI deployments, and his diagnosis is sharp: “Teams invest weeks building complex AI systems, but can’t tell me if their changes are helping or hurting.” The feedback loop the Three Ways assume — change, observe, learn, repeat — doesn’t run if you don’t know whether your change helped.

There’s also a category of failure that traditional DevOps doesn’t have a name for. An agent can return a fast, schema-valid, fluent response that’s factually wrong. APM doesn’t see it, latency is green, error rate is green. The user gets a hallucinated answer and either acts on it or never trusts your product again. Kim’s “unplanned work” — the predator that eats planned work — used to mean pager-duty incidents. Now it means an entire class of silent failure that has no signal in the traditional monitoring stack.

And there’s the autonomy claim, which is genuinely incompatible with the framework. The whole point of “You build it, you run it” (Werner Vogels, ACM Queue, 2006) is that a human team owns the page when their system breaks. The marketing around “self-improving agents” implies that ownership transfers to the agent. If the agent is improving itself, who’s getting paged when it ships a regression?

So that’s the steelman. The pipeline runs backwards. The feedback loop is broken. There’s a new class of silent failure. The autonomy claim breaks the accountability premise. The empirical data is pointing the wrong way. The framework should be dead.

It isn’t. But it took me a re-read of the book — paying attention to the flow parts rather than the war-story parts — to see why.

The artefacts changed

Here’s what I noticed on this re-read that I’d missed the first three times. Kim isn’t writing about code. He never was. Open The Phoenix Project to any random page and you’ll find Brent dealing with a payroll system, or Bill arguing about an SAN, or Erik explaining a heat-treat oven from a manufacturing plant. The book is studiously indifferent to what the work is. It cares only about how the work moves.

That makes the framework portable. Every artefact in modern AI engineering is just a substitution of the manufacturing-and-then-software-engineering versions Kim already wrote about.

Code became harnesses. When you ship an agent, the thing you write isn’t the model. The model is a dependency. The thing you write is the harness — the prompt, the tool definitions, the retrieval logic, the routing rules, the retry-and-reflection loop, the guardrail wrapping. It’s where the work lives. Improving an agent’s reliability is less about improving the model and more about improving the harness. Treat the harness like the system Kim cared about, and the rest falls into place.

Tests became evals. A traditional test is binary: did the function return the expected value? An eval is statistical: across a representative dataset, did the agent’s behaviour stay within an acceptable distribution of correctness? You author them differently — evals tend to be LLM-as-a-judge prompts or code-based checks running against curated examples — but they do the same job. They’re the executable specification. The deployment pipeline blocks a merge on a failing test suite; an eval pipeline blocks a merge on a failing eval run. Same gate. Different statistic.

Telemetry became traces. This is the cleanest mapping. Production agent systems emit spans for every LLM call, tool call, retrieval, reasoning step, and multi-agent handoff. Group those spans by session and you have the unit of work — the agent’s equivalent of a deployment. The instrumentation contract is OpenInference, which is OpenTelemetry semantic conventions extended for GenAI. The principle Kim’s Second Way rests on — “if it moves, we track it.” — is back, just attached to a different kind of moving thing.

Releases became eval-gated promotion. A traditional deployment pipeline runs tests, blocks on failure, promotes on green. An agent pipeline runs evals against a curated dataset, blocks on threshold failure, promotes on pass. The Definition of Done shifts from “all tests pass” to “this candidate beats the baseline on the dataset that encodes what the system should do.” A prompt change is a candidate. A model swap is a candidate. A retrieval re-index is a candidate. They all run the same gate. They all promote the same way.

Incidents became regression dataset entries. This is where the postmortem ritual genuinely breaks down — and we’ll come back to that — but the substitution itself is clean. A traditional incident produces a timeline, a contributing-factors list, and a set of corrective actions. With an agent, the artefact that prevents recurrence is a regression example — the specific input that failed in production, frozen into the eval dataset so the next candidate change has to handle it. The reproduce-and-prevent work the postmortem ritual does for systems is now done by the dataset.

Daily work became improving the system that improves the agent. Kim’s slogan from the novel: “Improving daily work is even more important than doing daily work.” In an agent shop, daily work is the agent’s runtime behaviour. The work that improves daily work is everything one layer up — adding new evals when failure modes appear, expanding regression datasets, calibrating judges, updating prompts, tightening guardrails. The work that improves that is one layer up again. The Third Way wasn’t about a specific kind of improvement work. It was about valuing that work as work.

Once you stop reading the book as being about code, the artefact replacements become obvious — and what’s left underneath them is a framework that survives the swap.

The architecture did not

Look at the substitutions together and what’s underneath them is the same shape. The Three Ways are still there. Different artefacts, same flow.

Flow. Work still moves left to right. A change to a prompt, a model, or a tool definition needs to make it through experimentation, evaluation, and promotion to production before it produces value. “until code is in production, no value is actually being generated” — Kim wrote that in 2013, and the only word that’s stale is “code.” Replace it with “prompt” and the sentence is identical.

Feedback. Production tells you what went wrong. Online evaluators run continuously against live traces, raising alerts when faithfulness or tool-call accuracy crosses a threshold. Charity Majors saw this in 2023; Kim saw it in 2013; John Allspaw at Etsy had articulated the blameless-postmortem version of it in 2012. The mechanism is older than the noun.

Continual learning. Kim extended his own framework into the AI era in Vibe Coding (2025, with Steve Yegge). His unifying principle for AI-assisted work — “Prevent problems, detect issues early, correct course quickly.” — is the Three Ways with the word “AI” added. The application needed an extension; the framework didn’t.

What survives across all three is the part Kim was most insistent on. “You build it, you run it” (Vogels, 2006) is the precondition. Twenty years later, Kim restates it in Vibe Coding: “Delegation of implementation doesn’t mean delegation of responsibility. Your users, colleagues, and leadership don’t (or shouldn’t) care which parts were written by AI — they rightfully expect you to stand behind every line of code.” Same principle, two decades apart. The agent is a tool, not a colleague.

That’s the part of the framework that didn’t need to change. It was already the abstract version.

The constraint moved

The most useful idea in the entire Phoenix Project isn’t the Three Ways. It’s the bit about Brent.

If you’ve read the book you remember Brent — the engineer through whom every change ends up being routed. Every incident pulls him into firefighting. Every “improvement” elsewhere just queues more work behind him. Erik teaches Bill that “Any improvements made anywhere besides the bottleneck are an illusion.” — borrowing directly from Goldratt’s Theory of Constraints. The framework’s whole operational lever, the thing it tells you to do, is: find your Brent, exploit his capacity, subordinate everything else to him, elevate him, then go find the next constraint because it’ll have moved.

Agent teams have a Brent too. He’s just not a person anymore.

Here’s the move that took me the longest to see. AI did not remove work centers from the value stream. It added stochastic ones. Codegen agents are work centers. Eval gates are work centers. LLM-as-judge evaluators are work centers. Labelling queues are work centers. Datasets are work centers. Guardrails are work centers. Harnesses are work centers. Each has a queue, a throughput rate, a setup cost, and a rework rate. What changed is that several of those work centers are probabilistic. Their outputs vary even with the same inputs.

This is the mechanism behind DORA’s 2024 finding. AI adoption raises throughput at the work centers that got AI — codegen, drafting, summarisation. It raises rework at the work centers AI feeds into — review, testing, production support. If the rework rate goes up faster than the throughput, delivery stability degrades. That’s the −7.2% number, in operational terms. The framework didn’t fail. It predicted exactly this. Kim is explicit that improvements not made at the constraint are illusions, and dumping output volume into a downstream work center that can’t process it is the textbook anti-pattern.

So who’s Brent now? On most agent teams I’ve seen, he’s one of:

The dataset. Your golden and regression sets are too small to catch real production failure modes. Evals pass; production fails. Elevating means investing in dataset curation — promoting failed traces into the regression set, expanding coverage, getting humans to annotate the edges. Not in shipping faster.
The annotator. Labelling queues back up. You can’t train an LLM-as-judge because you don’t have enough labelled disagreement data. You can’t expand the golden set because nobody’s looking at the candidate examples. Elevating means hiring SMEs, or shifting work to synthetic-dataset generation, or both.
The evaluator. The judge has drifted. Or the code-based check is measuring the wrong thing. Or the eval is so loose that everything passes. Elevating means investing in evaluator authoring and calibration, not in candidate-change throughput.
The harness. The agent is misbehaving because of how the routing or the planning or the tool definitions are structured, not because the model or the prompt is wrong. Adding evaluators won’t move this metric. The constraint is one level up.

The bottleneck moved. Kim told you it would. Step 5 of Goldratt’s Five Focusing Steps is “go back to step 1” — because the constraint always moves once you elevate it. The corollary every agent team underestimates is that the constraint will move again once you’ve fixed the dataset, and again once you’ve fixed the evaluator, and again once you’ve fixed the harness. A working agent program is one that knows which constraint it is currently working on.

DORA’s 2025 report sharpens this further. “AI’s primary role is as an amplifier, magnifying an organization’s existing strengths and weaknesses.” That’s exactly the Phoenix Project’s argument restated for the AI era. Teams that already have flow discipline, feedback loops, and constraint awareness get faster. Teams that don’t, get pulled apart by their own bottlenecks at higher speed. The framework didn’t fail. It became more load-bearing.

That’s the part of the book that aged the best. Not the chapters about flow, or feedback, or learning. The bit about Brent.

Where Phoenix needs repair

The framework holds up, but it’s not lossless. Three places where applying it 1:1 to agents would actually mislead you.

The pipeline gate is probabilistic, not deterministic. Kim’s “no defects pass downstream” rule assumes a binary test suite. The eval gate is an LLM-as-judge or a code-based check producing a score distribution against a sampled dataset. Gates pass at thresholds, not at green. The rule becomes “no statistically significant regression passes downstream,” which is contestable in ways the original wasn’t — what’s the sample size, what’s the threshold, what’s the judge’s calibration drift. Charity Majors put the sharper version in 2023: “With software, you typically start with tests and graduate to production. With ML, you have to start with production to generate your tests.” Production is the source of the test suite, not the destination of it. The First Way still works, but the gate it relies on is statistical, not binary.

“Self-improving” is not autonomy. The loop today is a human-in-the-loop loop with automation that progressively narrows the human role. Without the human-authored gates, an agent isn’t improving — it’s just changing. The framework is fine with this; what would break the framework is the autonomy claim itself. Vogels in 2006: “You build it, you run it.” Kim in 2025: “Delegation of implementation doesn’t mean delegation of responsibility.” The principle scales down from service to agent without changing. Any “self-improving” framing that implies the human stops owning the production loop is doing rhetoric, not engineering.

Blameless postmortems and the improvement kata don’t transfer cleanly. Both rituals were designed around the human loop — give engineers room to learn from failure without fear of blame; give them a structured daily routine for improving daily work. Apply them to agents and the rituals don’t fit. There’s no agent to be blameless toward. There’s no daily coaching session with an LLM. The Third-Way substance survives — failed traces become regression entries, daily work includes building the systems that build better systems — but the rituals don’t. Don’t run a blameless postmortem on an agent failure. Run it on the team that owns the agent. The agent goes into the regression dataset.

A whole category of observability and eval tools now ships this loop as product — distributed tracing, online and offline evaluators, dataset curation, experiment-gated CI. Arize Phoenix is the open-source version of it; Arize AX is the commercial one. The framework was waiting for the products to catch up.

What I’d do Monday

If you’re building agents in production, here’s what the re-read leaves me wanting to tell you.

Instrument everything as traces. Spans for every LLM call, tool call, retrieval, and reasoning step. Sessions for multi-turn work. Use OpenInference so you’re not locked in. The Second Way doesn’t work without telemetry, and telemetry doesn’t work without a contract that survives your vendor choice.

Promote production failures into your dataset. When something goes wrong in prod, the artefact that prevents recurrence isn’t a JIRA ticket. It’s the failed trace, frozen into your regression set, so the next change you ship has to handle it. The dataset is your Definition of Done.

Gate changes on evals in CI. Treat a prompt change or a model swap the same way you’d treat a code change. Run the eval suite. Compare against baseline. Block the merge on a failing gate. The pipeline mechanic is unchanged from the deterministic era; only the test type is different.

Find your current Brent. Walk the work centers. Where’s the queue? Is it the dataset waiting for annotation? The evaluator drifting out of calibration? The harness that needs restructuring? Wherever it is, that’s the only work that moves the bigger metric. Everything else is the illusion Kim warned about.

Run the loop, then improve the loop. Then improve the system that improves the loop. The Third Way isn’t a checkpoint you hit. It’s the standing instruction.

The book was about flow through a constrained system. The constrained system is now an agent fleet. The flow is the same.

It still holds up.

Train, dev, test: the split that makes an LLM judge trustworthy

Wed, 08 Jul 2026 00:00:00 +0000

Say you’ve built an LLM judge and you want to know if it’s any good. The obvious move is to feed it some labelled examples, tweak the prompt until its scores match your labels, and then point to those matching scores as proof it works.

Except you’ve proved nothing. You tuned the judge on those exact examples, so of course it agrees with them - you bent it until it did. It’s the oldest mistake in machine learning: marking your own homework with the answer sheet open. The fix is just as old, and it carries straight over to building evals. Split your labelled traces into three piles, and never let them touch.

Want it to hand? Download the infographic as a PDF.

The three piles run left to right from teach to validate. The rough proportions are on the cards - a small training slice, a medium dev slice, and the bulk held back for the test - and those proportions matter, so I’ll come back to why. First, what each pile is actually for.

Train, 10 to 20%

The smallest pile. This is the handful of hand-labelled traces you use to teach the judge what good and bad look like - ideally a few clear examples of a passing case and a few of a failing one.

These are the cases you drop straight into the prompt as few-shot examples. They’re the judge’s reference points, the “here’s what I mean” samples it pattern-matches against when it sees something new. (This is the examples slot from the anatomy of an eval prompt post - now you know where those examples are supposed to come from.) You don’t need many. Three to five passes and three to five fails is plenty, which is why this slice is small. Their job is to demonstrate, not to cover every case.

Dev, 20 to 30%

The middle pile is your workbench. This is the held-out set you iterate the prompt against, run after run, as you tune it.

You score the dev set, see where the judge disagrees with your labels, adjust the wording of the criteria or the rubric, and run it again. Tighten, re-score, repeat, until the judge’s scores line up with yours across the set. This is the loop where the actual work happens, and it’s deliberately not the train set - if you only ever checked against the few examples baked into the prompt, you’d just be confirming the judge can parrot back what you showed it. The dev set tells you whether it generalises a little beyond the exact cases it was taught.

Test, 50 to 70%

The biggest pile, and the one with the strictest rule: you don’t look at it. Not while you’re tuning, not to peek, not even once. It sits untouched the entire time you’re working on the dev set.

These are unseen labelled traces the judge has never encountered, and you score them exactly once, at the end, when you think you’re done. That single run is your real evidence. Because the judge was never tuned against these, its agreement with your labels here is an honest estimate of how it’ll do on live traffic it’s never seen. You measure the true-positive and true-negative rates - how often it correctly catches the bad cases and correctly passes the good ones - and that’s the number you actually trust. If you’d glanced at the test set while tuning, even a little, it’d be contaminated, and you’d be back to marking your own homework.

Why the proportions are upside down

Here’s the bit that catches people out. The split is backwards from what your instincts say. You teach with the smallest slice and validate with the largest, when intuition screams it should be the other way round.

But think about what each pile is for. Teaching needs only a few clear examples - a handful of passes and fails is enough to anchor the judge. Proving it works needs as much unseen data as you can spare, because the more cases you test on, the more you can trust the result. So the bulk of your data goes into the test pile, where it does the most good. A few examples teach; lots of examples prove.

That’s the whole trick, and it’s why an eval built this way is one you can actually defend. You taught the judge on one slice, tuned it on another, and proved it on a third it never saw coming. The score it gets on that final untouched pile is the one you can take to live traffic with a straight face - which, when the whole point of an eval is knowing it agrees with a human, is exactly the number you were after all along.

Evals do three jobs, not one

Mon, 06 Jul 2026 00:00:00 +0000

Ask most people what an eval is for and you’ll get some version of “testing”. You run it before you ship to check the app works, like a unit test. That’s true, and it’s also about a third of the story.

Evals do three different jobs, and the testing one is just the first. The same basic machinery - judge an output, score it - gets pointed at three completely different problems depending on when it runs and what happens next with the result. Treat all three as “testing” and you’ll under-use the other two, which is where a lot of the value actually is.

Want it to hand? Download the infographic as a PDF.

The three run left to right from before release to in production. The eval itself can be near enough identical across all three - same prompt, same judge. What changes is where it sits in the lifecycle and what you do with the score. That’s the bit worth getting straight.

Pre-release testing

This is the one everyone already knows. Before you ship, you run your evals over a golden dataset and check the scores clear the bar. It’s eval-driven development, and it works exactly like the unit tests you already write, just with a fuzzier kind of assertion at the end.

The value here is the same value tests have always had: you catch the regression while it’s cheap. A prompt change that quietly broke the tone of every reply, a model upgrade that made answers less grounded - you find it on your own dataset, before a single user is exposed to it. It’s the cheapest possible place to catch a problem, because the only thing that’s been harmed is a test run.

Production monitoring

Here’s where it gets interesting, and where most teams stop too early. Once the app is live, the evals don’t retire. They keep running, scoring real traffic as it flows through.

This matters because real users don’t behave like your test set. They ask things you never thought of, in combinations you never tried, and the world drifts out from under your carefully tuned golden dataset the moment you ship. Pre-release testing tells you the app worked on the inputs you imagined. Monitoring tells you whether it’s working on the inputs users actually bring, right now. You score live traffic continuously and you alert when quality drops, so you find out a model’s having a bad day from a dashboard rather than from an angry post going round. Same eval as the testing step, pointed at production instead of a fixed dataset.

Inline guardrails

The third job is the most aggressive. Here the eval runs inside the request path, checking a response in real time before it’s allowed out the door. And critically, the score doesn’t just get logged - it gets acted on, immediately.

If the check fails, you can block the response, regenerate it, or fall back to something safe, all before the user sees a thing. This is the eval as a bouncer rather than an auditor. The trade-off is that it has to be fast and it has to be cheap, because now it’s sitting in the critical path of every single request, adding latency to real user traffic. So you reserve guardrails for the things that genuinely can’t be allowed through - leaking personal data, going wildly off-topic, saying something unsafe - rather than every quality nicety you’d happily check after the fact.

Same tool, three jobs

The thing to take away is that “eval” isn’t one activity. It’s one tool doing three jobs, and the difference between them is timing and consequence. Pre-release, you test against a fixed set and block the release if it fails. In production, you monitor live traffic and alert if it dips. In the request path, you guard each response and act on it in real time.

Most teams build the first, occasionally build the second, and forget the third exists. But an app you can actually trust in front of real users usually needs all three - the test to ship with confidence, the monitor to know it’s still working, and the guardrail to catch the things that must never get out. It’s the same reason evals belong at every stage of the lifecycle and not just before launch: the work isn’t done when you ship, it just changes job.

Anatomy of an evaluator: what happens when your prompt meets your traces

Fri, 03 Jul 2026 00:00:00 +0000

Last time I pulled apart the eval prompt - the role, criteria, rubric and examples you write to tell an LLM judge how to judge. But a prompt on its own doesn’t do anything. Something has to take it, point it at your actual trace data, run it, and hand you back a result you can do something with.

That something is an evaluator. And the cleanest way to think about it is as a function: three inputs go in, three outputs come out. The prompt is just one of the three inputs. Understanding the other moving parts is what takes you from “I wrote a clever prompt” to “I have a number on a dashboard I can trust”.

Want it to hand? Download the infographic as a PDF.

The infographic runs left to right from inputs to outputs. On the input side sits the prompt I covered last time, plus the two pieces below: the data it reads, and the model that runs it. On the output side is the structured result. Get the inputs right and the outputs more or less take care of themselves.

Data fields

Your prompt talks about “the response” and “the retrieved context”, but those are just words until something fills them in with real data. That’s what data fields do. The prompt names the bits of a trace it needs as placeholders, and the evaluator binds each placeholder to an actual span attribute at run time.

This is the join between your nicely written prompt and the messy reality of a trace. You write the prompt once, with placeholders, and the evaluator drops in the real input, output and context for every span or trace it runs against. Get the bindings right and the judge is always looking at the actual data for the thing it’s grading. Get them wrong and you’ve got a beautifully worded prompt judging the wrong field, which is worse than no eval at all because it looks like it’s working. This is also why getting your tracing in order matters so much - the evaluator can only bind to spans you actually captured, which is the whole point of understanding spans, traces and sessions before you start.

Eval model

Then there’s the model that actually does the judging. The prompt and the data tell it what to assess; the eval model is the engine that reads all that and produces the verdict for each span or trace it runs against.

The thing worth noticing here is that this is a dial you can turn on its own. Your prompt and your data bindings don’t change when you swap the model. So you can run a cheap, fast model for a high-volume check where you just need a rough signal, and a stronger, pricier one for the judgement calls that really matter, all without rewriting a word of the prompt. A lot of the flexibility in the whole setup lives in this one input. It’s also where you make the speed-versus-cost trade-off that runs through every way of running an eval.

Output

Finally, the result. Each run hands back up to three things: a label (the categorical verdict, like correct or incorrect), an optional score (a number, if your rubric produces one), and an explanation (the model’s reasoning for why it landed where it did). Those get written back onto the span as structured eval attributes, sitting right alongside the data they were judging.

That structure is the whole reason this is useful rather than just interesting. Because the label is constrained and machine-parseable - not a free-form paragraph - you can aggregate it across thousands of traces, filter to just the failures, alert when the pass rate drops, and watch quality trend over time. The explanation is there for when you need to understand a specific call. The output isn’t a verdict you read once and forget; it’s data you can build dashboards and alerts on top of.

Where the flexibility lives

Step back and the shape is worth holding onto: prompt, data, and model go in; label, score, and explanation come out. The same three-in, three-out function, however simple or fancy the check.

And here’s the part that makes it click. Almost all the power and flexibility lives in those inputs. Want a stricter judge? Tighten the prompt. Want it judging a different part of the trace? Change the data bindings. Want it cheaper or sharper? Swap the model. The outputs stay the same clean, structured shape no matter what you do on the way in, which is exactly what lets you change how you judge without breaking everything downstream that depends on the result. Write a good prompt, point it at the right data, pick a fitting model, and the evaluator turns all that into something you can actually act on.

Anatomy of an eval prompt: what to actually put in it

Wed, 01 Jul 2026 00:00:00 +0000

When people decide to use an LLM as a judge, the prompt they reach for first is almost always some flavour of “rate this response from 1 to 10 on quality”. Then they’re surprised when the scores are all over the place and don’t agree with anything a human would say.

The reason is simple. That isn’t a prompt, it’s a coin toss with extra steps. You haven’t told the model who it’s meant to be, what “quality” means, what the numbers stand for, or what a good answer looks like. It’s filling in all of that itself, differently, every time you run it. A judge is only as good as the prompt behind it, and a good prompt has four parts.

Want it to hand? Download the infographic as a PDF.

The four parts run left to right from who is judging to how they score. Each one removes a little more of the model’s freedom to guess, and that’s the whole game. A reliable judge is one you’ve left as little room as possible to improvise.

Role

Start by telling the model who it is. Not “you are a helpful assistant”, but the specific kind of expert who’d be the right person to assess this particular output. You are an experienced support-quality reviewer. You are a senior engineer reviewing a code explanation for accuracy.

This feels like window dressing and it isn’t. A model with no role defaults to being a generic, agreeable, average responder, and average responders are soft markers - they want to give everyone a 7. Naming the role steers it to reason like the right expert, with the standards that expert would actually hold. You’re picking who shows up to do the judging, and it changes the answers you get.

Criteria

Now spell out exactly what you’re judging, in plain language. Not “quality”. The actual thing. Is this response factually correct given the retrieved context? Does it answer every part of the question the user asked? Is the tone appropriate for a frustrated customer?

This is the part that does the most work, and it’s the part people most often wave at. Vague criteria give you vague scores, every time. If you can’t say precisely what you’re checking for, the model certainly can’t, so it’ll invent its own definition and grade against that. The more concrete and explicit you are about what good means, the more reliable the judge becomes. This is the same “decide what good looks like” discipline that the whole build-an-eval process is built around, applied to a single prompt.

Rubric

Criteria tell the judge what to look at. The rubric tells it how to score what it sees. Define the scale and, crucially, what each level on it actually means. What’s a pass? What’s a partial? What’s an outright fail, and what specifically tips a response from one into the next?

Without this, a 1-to-10 score is meaningless - the model’s 6 today is its 8 tomorrow, because nothing anchors them. A rubric with described levels turns a fuzzy gut feeling into something repeatable. Two runs over the same response should land on the same score, and two different responses with the same problem should score the same way. That consistency is the entire reason you’re scoring things in the first place, and the rubric is what buys it.

Examples

Last, show the judge a handful of real cases with the labels you’d give them. A couple of clear passes, a couple of clear fails, drawn from your own data - the actual responses your actual app produced, judged the way you’d judge them.

These few-shot examples are the strongest lever of the four. A model left to its own devices judges against its own priors, which are some average of the entire internet. Show it three good and three bad examples from your domain and it stops guessing and starts pattern-matching to your standard instead. This is also exactly where a proper train/dev/test split earns its keep: the examples come from your training slice, so you’re never showing the judge a case you later use to score it.

The pattern underneath

Look at the four parts together and there’s one idea running through all of them: every part you add takes away a little of the model’s freedom to make things up. Role decides who’s judging. Criteria decide what they’re judging. The rubric decides how they score it. Examples show them exactly what you mean. Each one closes off another way for the judge to drift.

That’s the real anatomy of a good eval prompt. Not “rate this 1 to 10”, but a tight brief that leaves a clear, specific, expert reviewer almost nowhere to improvise. Write that, and the scores start agreeing with you - which is the only thing that makes a judge worth running at all. The prompt is half the picture, though. The other half is what your eval platform does with it, which is a post of its own.

Spans, traces and sessions: the three zoom levels of an AI app

Mon, 29 Jun 2026 00:00:00 +0000

The moment you start tracing an AI app, three words turn up everywhere: span, trace, session. People sling them around as if they’re obviously different, and if you’re new to this you nod along while quietly wondering whether they’re just three names for the same thing.

They’re not. They’re three zoom levels on the same activity, from a single atomic step all the way out to a whole conversation. And the reason it’s worth getting straight is that you evaluate something different at each level. Point an eval at the wrong zoom level and you’ll get an answer to a question you didn’t mean to ask.

Want it to hand? Download the infographic as a PDF.

The whole thing runs left to right from atomic, a single step to the whole conversation. Think of it like zooming out on a map. A span is one building. A trace is the street it’s on. A session is the whole neighbourhood. Same place, different altitude, and you ask different questions depending on how high up you are.

Span

A span is the smallest unit: one atomic operation. One model call. One tool call. One retrieval step. The single thing the app did, on its own, with a start and an end.

When you evaluate at the span level, you’re asking a tight, specific question about that one step. Did the retrieval actually pull back relevant documents? Did the tool return the right result for the arguments it was given? Did this one model call produce sensible output? It’s the most precise place to look, because there’s nowhere for a problem to hide - it’s just the one operation, in isolation.

This is where a lot of the most useful debugging happens. When a whole answer comes out wrong, the cause is usually one bad span buried in the middle - a retrieval that grabbed the wrong thing, a tool that quietly failed. Evaluating at the span level is how you find the exact step that let you down.

Trace

Zoom out one level and you get a trace: all the spans for one user-facing request, strung together. The user asks a question, and the app does a whole sequence of things to answer it - retrieves some context, calls a model, maybe calls a tool, calls the model again - and the trace is that entire path from input to final answer.

Evaluating at the trace level asks the bigger question: did the whole turn actually land? Not “was this one retrieval good”, but “given everything the app did, was the final response grounded, correct, and on-task?” This is the level that maps to what the user actually experienced for that one question. Every span in it could look individually fine and the answer could still miss, which is exactly why you need to evaluate the trace as a whole and not just its parts.

Session

Zoom out once more and you’ve got a session: a collection of traces making up a whole multi-turn conversation. The full back-and-forth, from the user’s first message to the point they leave.

This is the level people forget about, and it’s where some of the most important failures live. A single turn can be perfect while the conversation as a whole falls apart. The app contradicts something it said three messages ago. It loses the thread. It technically answers every individual question but never actually helps the person get where they were going. Evaluating at the session level asks the only question that ultimately matters: did the user reach their goal? You’re looking at coherence across turns, whether things actually got resolved, and the tell-tale signs of a frustrated human going in circles.

Pick the right altitude

So when you sit down to evaluate, the first question isn’t “what’s my eval” - it’s “at what level am I asking?”

Checking whether a retrieval step did its job is a span-level question. Checking whether a whole answer was grounded is a trace-level question. Checking whether the user actually got helped is a session-level question. Same app, three different altitudes, three different things worth knowing.

Most teams start and stop at the trace level, because that’s the obvious one - it maps to a single question and answer. But the span level is where you debug why something broke, and the session level is where you find out whether you’re actually helping anyone over the course of a real conversation. Get comfortable moving between all three, and the question of what to evaluate gets a lot clearer. The next thing to sort out is how to run those evals once you know what you’re asking, which is a whole spectrum of its own.

Evals aren't a step at the end. They run the whole way through

Fri, 26 Jun 2026 00:00:00 +0000

There’s a version of building an AI app that goes like this. You build the thing, you get it mostly working, and then someone says “should we evaluate it?” and you bolt some evals on at the end like a spoiler on a hatchback. It runs, the numbers look fine, you ship.

That works about as well as writing all your tests the night before launch. Evals aren’t a stage you do once at the end. They’re something that runs the whole way through, doing a different job at each step. Get that idea straight and the rest of this falls into place.

Want it to hand? Download the infographic as a PDF.

The whole thing runs left to right, from build and pre-release over on the left to production and live on the right. Same lifecycle you already know from normal software. The difference is that an AI app is non-deterministic, so “does it work” isn’t a yes or no you can answer once. You have to keep asking, at every stage, and an eval is how you ask.

Requirements

Before you write a line of code, you have to decide what the app is actually for and what a good answer even looks like. This sounds obvious and almost everyone skips it.

The trap with AI is that “good” feels self-evident until you try to pin it down. A good support reply is… helpful? Polite? Correct? Grounded in the actual docs? Those are four different things, and an answer can nail one and fail the others. So this is where you turn fuzzy intentions into concrete success criteria, and start gathering the examples that show what good looks like. That collection becomes your golden dataset, which everything downstream gets measured against. I wrote a whole post on building an eval you can trust, because it’s the foundation the rest sits on.

Design

Now you choose how to build it. Which prompts, which model, what architecture. And here’s where most people go on vibes - they try a prompt, it looks good in the playground, they ship it.

The better move is to make those decisions on evidence. You’ve got a golden dataset from the last step, so use it. Run two candidate prompts against it and see which actually scores higher. Compare a cheaper model against an expensive one on the cases that matter to you, not on a leaderboard built from someone else’s data. Design becomes a series of small head-to-head bake-offs instead of a series of hunches, and you end up committing to choices you can defend.

Implementation

You build the app. You wire up the tools, the retrieval, the integrations, all the moving parts. Nothing surprising here, except for one thing you have to do that you might not be used to: instrument it.

Tracing isn’t optional for an AI app. When something goes wrong - and it will - you need to see what the app did, step by step, to figure out why. That means every span, every trace, every session is captured and inspectable. If you don’t have that, debugging an AI app is just guessing with extra steps. (Span, trace and session are the three levels you’ll be looking at, and they’re worth understanding properly - here’s a post on exactly that.) Get the instrumentation in now, while you’re building, not after you’ve shipped and you’re trying to bolt it on under pressure.

Testing

Now you validate before you ship, the same way you’d unit-test normal code. The twist is that you can’t assert on an exact output, because the output changes every run. So the shape of the test changes too.

Normal testing goes arrange, act, assert. You set things up, you run the code, you check the result is exactly what you expected. Eval-driven development adds a step: arrange, act, evaluate, assert. You set things up, you run the app, you run an eval on whatever came back, and then you assert on the eval’s score. You’re not checking the output equals a fixed string. You’re checking the output scores above the bar you set in the requirements. Same testing instinct, adjusted for a world where the answer is never byte-for-byte the same twice.

Deployment

You ship, and your app meets real users. Real users are messier than anything you tested with. They ask things you never imagined, in ways you never anticipated, and the neat distribution of inputs you tuned everything against goes out the window.

So the evals don’t stop at the door. They follow the app into production. You run online evals on live traffic, scoring real responses as they go out, and you alert the moment quality drops. This is the part that turns “it worked when we tested it” into “it’s still working right now”, which is the only version that actually matters once people depend on it.

The thread that runs through it

Notice what carries the whole way along. The golden dataset and the success criteria you defined at the very start are the same yardstick you use to compare designs, to test before shipping, and to judge live traffic. Define “good” once, properly, and it pays off at every stage after.

That’s the real point. Evals aren’t a gate at the end that says yes or no. They’re a feedback loop you run continuously, from the first requirement to live production traffic, and the job they do shifts as you move along. Bolt them on at the end and you’ll catch the odd bug. Thread them through from the start and you actually know, at every step, whether the thing works - which, for something as slippery as an AI app, is about the most valuable thing you can know.

Four ways to run an eval, from a cheap unit test to a full-blown agent

Thu, 25 Jun 2026 00:00:00 +0000

Someone asked me last week how you actually run an eval on an AI app. I gave the honest answer, which is “it depends”, and then watched their face do the thing faces do when you give them the honest answer. So let me give the longer version, because “it depends” is true but useless on its own.

There isn’t one way to run an eval. There’s a spectrum. At one end you’ve got a check so simple it’s basically a unit test. At the other end you’ve got an agent crawling through your traces like a detective. And the interesting bit is the middle, which most people skip straight past.

Want it to hand? Download the infographic as a PDF.

The whole thing runs left to right from deterministic, fast and cheap to agentic, flexible and powerful. As you move right you gain intelligence and nuance, and you pay for it in speed, money, and the loss of a thing that turns out to matter a lot: determinism. Run the same check twice on the left, you get the same answer twice. Do it on the right and you’re at the mercy of a model having a good day.

Here’s the rule I keep coming back to, and I’ll spoil the ending now: start as far left as you possibly can, and only move right when the question genuinely needs it.

Code

The cheapest eval is just code. A function that takes the output and returns pass or fail. Did the model return valid JSON? Does the response match the schema? Is there a phone number in there shaped like a phone number? Did it come back under two seconds?

It’s a unit test for your model. That’s not me being reductive - that’s genuinely what it is. Same tooling, same mindset, same satisfying green tick.

Code evals are fast, they’re basically free, they’re completely objective, and they give you the same answer every single time. The catch is they’re blind to anything fuzzy. A code check can tell you the response is valid JSON. It cannot tell you the response is rude, or unhelpful, or confidently made up. For that you need something that can read.

But here’s the thing - a huge amount of what you want to check is not fuzzy at all. Format, structure, presence of a required field, staying under a length limit. Use a code eval for every one of those, every time. Don’t pay a language model to count characters.

LLM as a judge

When the thing you’re checking is genuinely subjective, you reach for an LLM as a judge. You hand a model the output and ask it to score something a human would otherwise have to score - tone, helpfulness, factual accuracy, whether the answer is actually grounded in the source you gave it.

This sounds dodgy the first time you hear it. You’re using an AI to mark an AI’s homework? Yeah, and it works better than you’d expect. A well-prompted judge lands around 85% agreement with human reviewers, which is about as often as two humans agree with each other. Turns out we’re a noisy bunch.

The cost is real, though. Every eval is now an API call, so it’s slower and you’re paying per check. And a judge needs calibrating - you have to actually check it agrees with you before you trust it, which is a whole topic of its own (I wrote about that here).

So reserve the judge for the fuzzy stuff. The subjective questions code can’t touch. Don’t point it at things a regex would’ve nailed for free.

Code + LLM

This is the one people skip, and it’s the one I’d most like to talk you into.

Real traces are messy. A judge pointed at a raw trace has to wade through tool calls, retries, system prompts and metadata just to find the bit it’s supposed to be grading. That’s slow, it’s expensive, and it’s where judges get confused and start scoring the wrong thing.

So you do the boring part in code first. Code deterministically pulls out exactly the span you care about, parses the field you need, cleans up the input - and then hands that tidy little nugget to the LLM to judge. Code does the extraction, the model does the reasoning.

You get cheaper, more reliable evals out of it, because the model is only ever looking at the signal and never the noise. Think of code as the cheap pre-filter that does the donkey work before you pay for the clever bit. It’s a bit more plumbing up front, but on real production traces it’s the difference between a judge that works and one that quietly grades the wrong thing for a month.

Harness as a judge

The far right of the spectrum is the new and genuinely exciting one. Instead of a single judge call, you give an agent harness - something like Claude Code - the keys to your trace data.

It pulls traces through an API or a CLI, inspects spans, follows a thread when something looks off, and uses tools to go and check. It’s not answering one fixed question. It’s investigating. And because it can reason across a whole trace rather than one extracted field, it can spot things you didn’t think to write a check for. Point it at a pile of traces and it can even come back and suggest the evals you should be running.

The one hard requirement: your trace data has to be reachable. If it’s all locked inside a dashboard you can only look at, the agent can’t touch it and the loop stays manual. If it’s available through APIs, CLIs and standard formats, the agent can just get on with it.

This is the most flexible and most powerful option by a mile. It is also the slowest, the priciest, and the least deterministic - you’re handing the wheel to an agent and trusting where it drives. So it’s the thing you reach for when you want to understand what’s going wrong across your whole system, not the thing you run on every single response in production.

So which one?

All four. That’s the actual answer.

Start as far left as you can. Push everything that’s deterministic into code evals, because they’re free and they never lie to you. Bring in an LLM judge only for the genuinely subjective stuff code can’t see. When your traces are messy - and they will be - put code in front of the judge to do the extraction. And keep a harness in your back pocket for the bigger questions, the “what’s actually going wrong here” investigations that don’t fit in a single check.

The mistake I see most often is reaching straight for the expensive end. Someone writes a clever LLM judge for something a three-line code check would’ve handled, then wonders why their eval bill looks like that. Match the method to the question. Most of your questions are cheaper than you think.

One more thing before you run off and build any of this: an eval is only worth running if you know it agrees with a human. That part deserves its own post, so here’s that post.

I built a little Claude that dances when it needs me

Wed, 24 Jun 2026 00:00:00 +0000

I’ve got into a bad habit lately. I set Claude Code off on some task - refactor this, write tests for that - then I tab away to read Slack or stare out of the window, and the next time I look back the terminal has been sitting there for five minutes quietly asking “can I run this command?”. Claude was ready. I was not. Multiply that across a few sessions running at once and I’m losing little chunks of my day to not noticing.

So I did what any reasonable person would do. I built a tiny dancing robot to watch my back.

TLDR; It’s called claude-notify, it’s a little Claude character on a Raspberry Pi that dances when Claude Code is waiting for you. Code’s on GitHub: github.com/jimbobbennett/claude-notify.

What it actually is

Picture the little orange Claude sunburst - the flower-ish face with the rays coming off it. That character lives on a cheap Raspberry Pi touchscreen sat on the corner of my desk. While Claude Code is busy working, the screen is dark and the character slumps about looking bored - it yawns, it blinks, it sighs, it nods off with little zzz’s floating up.

Then the moment Claude Code stops and needs me - a permission prompt, a question, or just “I’m done, what next?” - the whole screen flips purple, the character springs up and starts dancing, and a label tells me which project it is. “YOUR TURN!”

It’s daft. It’s also genuinely useful, which I did not entirely expect.

How it hangs together

Here’s the bit that surprised me: the hard part - getting Claude Code to tell something else what it’s doing - turned out to be the easy part. Claude Code has hooks, little scripts that fire on events like “about to use a tool” or “stopped and waiting”. You wire them up in your settings and they get the event details as JSON on stdin. That’s the whole ballgame.

So the architecture is just a chain of dead-simple pieces:

Claude Code hooks → shell script → HTTP POST → Flask on the Pi → SSE → browser

Each link is boring on its own, which is exactly what you want. Let me walk the chain.

One script, not seven

Claude Code fires a bunch of different hook events, and I’m interested in most of them. Rather than write seven little scripts, there’s one script that takes a subcommand:

# in ~/.claude/settings.json, every hook points at the same script
hook-pi.sh notify # Stop / Notification → start dancing
hook-pi.sh idle # UserPromptSubmit → back to bored
hook-pi.sh heartbeat # PreToolUse / PostToolUse / SessionStart → still alive
hook-pi.sh end # SessionEnd → remove me

The script reads the hook’s JSON off stdin, pulls out the session_id and the current folder name with jq, and POSTs that to the Pi. That’s it. Cross-platform too - there’s a POSIX sh version for Mac, Linux and WSL, and a native PowerShell one for Windows.

The Pi just holds a dictionary

On the Pi there’s a tiny Flask server listening on port 8080. It keeps a dictionary of sessions keyed by session_id, and each incoming POST nudges one session into dancing or idle. There are a handful of endpoints - /notify, /idle, /heartbeat, /end - and one more, /events, that the browser connects to.

That last one is Server-Sent Events, which I’d somehow never used before this and now want to use for everything. It’s a one-way stream from server to browser over plain HTTP - no WebSocket handshake, no library, the browser just goes new EventSource('/events') and listens. Every time the state changes, the Pi pushes the whole snapshot down the pipe and the page redraws. For a “show me the current state” screen it’s perfect.

The character is hand-built SVG

There are no image files for the mascot - the whole thing is an SVG drawn in the page, animated with CSS keyframes. Rays, body, two eyes, little cheeks, a mouth. Dancing is one keyframe rocking it side to side; the bored acts (yawn, blink, sigh, sleep, tilt) are separate keyframes a scheduler triggers at random.

That scheduler is the one bit I’m a little smug about. With four bored mascots on screen you don’t want all four animating constantly - that’s a lot of work for a Pi driving a slow little SPI screen. So a single global timer picks one idle character every 6-15 seconds and gives it something to do. Cheap, and it reads as more lifelike than everything moving in lockstep.

The fiddly bits nobody warns you about

The happy path was a weekend. The edges took longer, as they always do.

Knowing when Claude is actually stuck. A dancing mascot should only dance when Claude is genuinely waiting. But what if it starts dancing and then I’m not looking and it just… keeps going? The fix falls out of the heartbeats for free: the tool hooks (PreToolUse, PostToolUse) fire constantly while Claude works. So if a heartbeat arrives, Claude is clearly doing something, not waiting - and that automatically downgrades a dancing mascot back to idle. The thing that proves it’s busy is the thing that calms it down.

Sessions that die without saying goodbye. There’s a SessionEnd hook to remove a mascot when a session closes cleanly. But if you kill -9 Claude, or just slam the terminal shut, that hook never fires - and you’re left with a ghost mascot forever. So there’s a little watchdog on the Pi that evicts any session it hasn’t heard from in ten minutes. Not elegant, but it means the screen self-heals.

It has to survive being ignored. This thing sits on a desk and is supposed to Just Work for weeks without me touching it. So the kiosk launcher is far more paranoid than the rest of the project: it waits for the display to come up, disables screen blanking, and restarts Chromium with exponential backoff if it ever crashes - backing right off if it’s failing repeatedly so it doesn’t thrash. Wildly over-engineered for a dancing cartoon. Absolutely worth it.

The whimsy words

One last thing, because it made me happy. While Claude is working, the mascot shows a little activity word under it. I could have shown the actual tool being run. I did not. Instead it picks from a list of nonsense - “Frobnicating…”, “Hornswoggling…”, “Galumphing…” - in the same spirit as Claude Code’s own status line. It does absolutely nothing useful and I wouldn’t remove it for the world.

Want to build one?

Everything’s on GitHub: github.com/jimbobbennett/claude-notify. You’ll need a Raspberry Pi that runs the desktop OS (a Zero 2 W is plenty) and one of those little SPI touchscreens, though honestly any spare screen will do. The installers merge into your ~/.claude/settings.json for you and back it up first.

And if you don’t fancy the hardware, the interesting transferable bit is the pattern: Claude Code hooks are an open door to making it talk to anything. A dancing robot is just the silliest possible thing I could point that door at. What would you point it at?

Happy making!

How to build an eval you can actually trust

Thu, 18 Jun 2026 00:00:00 +0000

Here’s how most people build an eval. They open a file, write an LLM judge prompt that says something like “rate this response from 1 to 10 on helpfulness”, run it over a few hundred traces, get a pile of numbers back, and then have absolutely no idea whether those numbers mean anything.

I get it. It feels like progress. You’ve got a dashboard, you’ve got a metric trending, you can put it on a slide. But you’ve skipped the part that makes an eval worth running, which is knowing it agrees with a human. A number you can’t trust is worse than no number, because at least no number doesn’t lull you into thinking everything’s fine.

So here’s the longer way round. It’s five steps, and it borrows a trick from qualitative researchers - the people who code interview transcripts for a living and have been doing the “turn messy human judgement into something measurable” thing for decades.

Want it to hand? Download the infographic as a PDF.

The whole flow goes left to right, from gather and observe to measure and validate. The first half is about understanding what actually goes wrong. The second half is about building checks for it and proving they work. Skip the first half and you’re just guessing in a more expensive way.

Step 1: Gather a golden dataset

You can’t measure anything without a set of examples to measure against. So the first job is to collect a representative set of real interactions your app has to handle - the dull ones, the weird ones, the ones that fall over.

The word that matters here is representative. Everything downstream gets measured against this set, so if it doesn’t look like real usage, your evals are tuned for a world that doesn’t exist. Pull from production traffic if you’ve got it.

If you don’t have enough real data yet - and early on, you won’t - synthesise some. Generate examples that look like the traffic you expect. It’s not as good as the real thing, but a synthetic golden set you’ve thought about beats no golden set at all. Just be honest with yourself about which bits are made up.

Step 2: Human open coding

Now the unglamorous bit. A human - probably you - sits down and reads each example in the golden set, and writes free-form notes on what went wrong. Not against a checklist. Not “score this 3 out of 5”. Just plain notes: “made up a refund policy”, “ignored half the question”, “weirdly passive-aggressive tone”.

This is called open coding, and the whole point is that you have no fixed categories yet. You’re not confirming a theory about what your app gets wrong. You’re letting the failure modes show up on their own.

This matters more than it sounds. If you start with a list of things to check, you’ll only ever find the things on your list. Open coding surfaces the failures from the ground up - including the ones you’d never have thought to look for, which are usually the ones that bite.

Step 3: Axial coding

Open coding leaves you with a big messy pile of notes. Step three is turning that pile into a tight set of categories. This is axial coding, and it’s where you let an LLM earn its keep.

You and the model go through the notes together and group them into recurring failure categories. Forty scattered observations collapse into something like five real buckets: hallucinated policy, incomplete answer, tone problems, ignored context, broke format. The LLM is genuinely good at this - spotting that “made up a refund policy” and “invented a returns window” are the same underlying failure - and you’re there to sanity-check that the buckets actually make sense.

What you get out of this is a taxonomy. And that taxonomy is the thing your evals will measure. You’ve gone from “I think our bot is a bit off sometimes” to “here are the five specific ways it fails”, which is a much better place to be standing.

While you’re here, tag each example in the golden set with the categories it actually shows - including the ones that are simply fine. That labelled set is your ground truth, and it’s what you’ll check your evals against in a minute.

Step 4: Design the evals

Now, and only now, do you build evals. One per failure category. You know exactly what you’re checking for, because steps two and three told you.

For each category, pick the cheapest method that’ll do the job. Broke format is a code check - free, fast, deterministic. Passive-aggressive tone needs an LLM judge, because no regex is going to catch attitude. Some categories want a bit of both. I went through the whole spectrum of how to run these - plain code checks, LLM judges, code-and-LLM hybrids, and full agentic harnesses - in a separate post on the four ways to run an eval, so I won’t repeat it here.

The point is that by this stage, designing the eval is the easy part. You’re not guessing what to measure. You’re building a known check for a known failure.

Step 5: Test against the golden set

This is the step everyone skips, and it’s the one that makes the whole thing trustworthy.

You’ve got your golden set. You’ve got human labels on it from steps two and three - you know what actually went wrong in each example. So run your shiny new evals back over that same golden set and check whether they agree with the human verdicts.

If your tone judge flags the same responses you flagged, brilliant - that’s real evidence it’ll hold up on traffic it’s never seen. If it disagrees, you’ve caught it now, on a controlled set, instead of six weeks into production when it’s been quietly grading the wrong thing the whole time. Tweak the eval, run it again, repeat until it lines up with the humans.

That’s the bit that turns a number into a number you believe. Your eval has been checked against ground truth before it ever touches live data.

Why bother with all this

Because the alternative is a metric that looks like signal and is actually noise, and those are dangerous - they’re confidently wrong, and they’re on a dashboard, so everyone trusts them.

The open-coding-then-axial-coding move is the bit worth stealing even if you ignore everything else here. Look before you measure. Let the failures tell you what the categories are, instead of deciding the categories first and forcing reality to fit. Researchers have been doing it for decades because it works, and it carries over nicely from a stack of interview transcripts to a pile of agent traces.

Build the golden set, read it properly, find the real failure modes, build a check for each one, and prove those checks agree with you. Then you’ve got an eval you can actually trust - and once you trust it, running it is the easy part.

Two labs started dreaming, and they built two different architectures

Wed, 17 Jun 2026 07:35:50 +0000

Originally published on the Arize AI blog: Two labs started dreaming, and they built two different architectures.

On May 6, 2026, Anthropic launched Dreams for its Managed Agents platform, the system for hosting stateful background agents. 27 days later, on June 2, OpenAI shipped Dreaming V3 for ChatGPT, the third iteration of a feature it has called “dreaming” since April 2025. By shipping under the same word, Anthropic adopted a technical term OpenAI had been using for 14 months. Both companies now treat the release as central to a multi-year arc Sam Altman has described publicly since 2025 as memory approaching the shape of remembering your whole life.

Dreaming, in the agent-memory sense, is the pattern where a background process reads past session transcripts and writes a consolidated memory artifact the next session can use. It has a 40-year lineage in machine learning, but the productized form is new. Underneath this shared word, Anthropic and OpenAI built two different architectures. The research that landed in the same window suggests the difference matters more than the convergence does.

This post is about what each lab actually shipped, what the empirical literature says about the consolidation pattern in general, and what to do if you are the team trying to figure out whether your version of dreaming is helping the agent or quietly degrading it.

What Anthropic shipped

Anthropic’s Dreams is the more cautious of the two designs, and its caution is the most interesting thing about it. Anthropic launched the feature as a research preview at its Code with Claude developers’ conference on May 6, scoped to Managed Agents.

The API is explicit. You call it asynchronously, hand it an existing memory store and between one and 100 raw session transcripts, choose a model, and gate the whole thing behind a beta API header. The dream produces a separate, reorganized memory store, with the input store left alone. The Anthropic documentation is unusually direct about this commitment:

“A dream reads an existing memory store alongside past session transcripts, then produces a new, reorganized memory store… The input store is never modified.”

The commitment is doing architectural work: Anthropic’s engineers expect the dream to sometimes degrade the memory rather than improve it, and they have built the system so that the comparison is possible. You can run an eval against the pre-dream store, run the same eval against the post-dream store, and decide whether to promote the new artifact or throw it away. The dream is an experiment, not a commitment.

The commit boundary is also a choice the team running Dreams gets to make. From Ars Technica’s reporting on the launch: “Users will be able to choose between an automatic process, or reviewing changes to memory directly.” You can let the system auto-promote the new store, or you can require human review before changes take effect. Either way the previous state is preserved.

Anthropic’s own framing of the value is cross-agent rather than per-user. Dreaming “restructures memory so it stays high-signal” across team-shared workflows. The interesting word is “restructures.” Anthropic believes the dream is constructive; they have just chosen to leave the previous structure intact so a team can fall back to it.

The system reads as if it were designed by people who had thought hard about what could go wrong.

What OpenAI shipped

OpenAI’s Dreaming for ChatGPT is the more ambitious of the two designs. Dreaming is not a new feature: “Dreaming V0” was introduced in April 2025 alongside saved memories, and what shipped on June 2 is Dreaming V3, described in the launch post as “a significantly more capable and compute-efficient memory architecture built on top of dreaming.”

OpenAI describes the mechanism in its own words: “dreaming leverages a background process that allows ChatGPT to learn from many conversations and synthesize ChatGPT’s memory state in order to always provide the freshest, most relevant context to your conversations.” The companion memory FAQ uses parallel language: “ChatGPT’s memory is based on a continually updated synthesis of context from your past chats.”

Both documents describe a single canonical state that gets rewritten as the system runs, not a parallel artifact alongside the existing one. “Continually updated synthesis” implies that each pass is built on whatever the previous one wrote, though OpenAI does not state this explicitly.

OpenAI presents Dreaming as solving temporal supersession. “You’re going to Singapore in July” should become “You went to Singapore in July 2026” once the trip ends, automatically, without manual intervention.

The user-facing controls are the closest thing the architecture has to an escape hatch. A memory summary updates roughly hourly (the FAQ documents an “Updated 2h ago” indicator), and users can edit individual entries or hide them. There’s also a settings toggle to revert to legacy saved memories.

Those controls operate on individual items, not on the synthesized state itself. The memory FAQ is explicit about the distinction: “details from past chats can change over time as ChatGPT updates what’s more helpful to remember. Because ChatGPT doesn’t retain every detail from past chats, use saved memories for anything you want it to always remember.” Saved memories (the explicit 2024 layer, with per-entry edit and restore-by-date) are the persistent surface. The Dreaming synthesis is the mutable one. The legacy-memory toggle switches between the two systems; it does not roll back a dream.

The summary itself is a partial view: the FAQ adds that it “will not include everything that ChatGPT remembers based on your chats.”

The system reads as if it were designed by people who believed the dreaming metaphor.

The divergence

Anthropic preserves the input store and produces a parallel artifact. OpenAI rewrites the memory in place and lets the next pass condition on the previous one’s output. Both companies use the same word for the feature, but they made opposite bets about whether the dream should be reversible.

This is not a small distinction. The empirical literature on iterated LLM-driven consolidation, which landed in the same window as both launches, suggests it is the most important distinction in the entire feature.

Why the divergence matters: Zhang’s paper

In May 2026, Dylan Zhang at the University of Illinois Urbana-Champaign published Useful Memories Become Faulty When Continuously Updated by LLMs (project page, arXiv 2605.12978). The paper is the cleanest empirical case yet made against the iterated-consolidation pattern, and the central finding is striking.

On ARC-AGI, the abstraction-and-reasoning benchmark, GPT-5.4 solves 19 problems at 100% accuracy without memory. After streaming those same problems through consolidation loops with ground-truth solutions available, accuracy drops to 54%.

The trajectories were perfect. The failure happened in the rewrite step. The act of compressing right answers into a re-usable lesson, Zhang argues, is what made the system forget how to solve them.

The pattern repeats across benchmarks.

On ScienceWorld, a text-based scientific reasoning benchmark, score peaks after roughly 20 updates and then declines below the no-memory baseline by step 100.
On WebShop, a simulated e-commerce task suite, performance falls from 0.64 with eight examples to 0.20 with 128, matching the no-memory baseline at scale.
On ALFWorld, an embodied household-task benchmark, three different solver sizes (Qwen 27B, 9B, and 4B) all show utility decay across consolidation steps.

Zhang identifies three reproducible failure modes inside the consolidation step:

Misgrouping – episodes from distinct classes merge into single entries
Interference – abstraction strips applicability conditions, so lessons generalize too broadly
Overfit – specific selectors get deleted as the input distribution narrows

The architectural diagnosis Zhang offers is precise:

“Continuously updated textual memory is an iterated generative loop with no anchor… it is a sample — fluent, confident, and increasingly disconnected from what actually happened.”

Two memory modes appear in Zhang’s experiments:

Episodic memory retains the raw rollouts: the actual session transcripts, unchanged, and the system retrieves them at recall time.
Abstract memory stores the LLM’s compressed summary of those rollouts, the lessons and generalized rules synthesized from them, and discards the originals.

Most production memory systems, including both Dreams and Dreaming, sit on the abstract end.

Zhang’s recommendation is architectural: episodic-only memory recovers nearly the entire gain consolidation-based variants give up. Abstract-only never beats the no-memory baseline. The implication is sharp: the abstraction step is the failure point, and both Dreams and Dreaming run it. Only OpenAI runs it iteratively against its own previous output, which is the specific pattern Zhang’s results target.

The paper is not a rejection of memory as a capability. It is a rejection of the specific pattern of letting an LLM rewrite its own memory in place across many iterations. That pattern is exactly what OpenAI’s Dreaming architecture does, while Anthropic’s Dreams architecture does not.

How the architectures meet the research

Anthropic’s design preserves the input store. That means a team running Anthropic Dreams can run an eval against the pre-dream store, run the same eval against the post-dream store, and discard the post-dream artifact if it scores worse. The dream is reversible. The iterated-generative-loop failure mode Zhang documents is constrained to whatever happens inside a single dream invocation; across invocations, the team chooses whether to keep what the dream produced or roll back to the previous state.

OpenAI’s design rewrites in place. The previous state is overwritten, and the per-item user controls (edit, “don’t mention this again”) are not a pre/post comparison surface. That shape is the one Zhang identifies as failing.

TechTimes’ coverage of the launch put the same point in user-facing terms: the redesign “limits the granular audit trail that the discrete saved-memories list provided.”

OpenAI’s framing creates a particular tension with Zhang’s findings. OpenAI claims Dreaming handles temporal supersession. Old facts get gracefully replaced as time passes, the Singapore trip going from upcoming to over to historical without manual intervention. Zhang’s mechanism suggests iterated consolidation is particularly bad at preserving the specific facts that need to be updated. The Singapore trip is exactly the kind of detail (a specific date, a specific place) that Zhang’s failure mode predicts will get rounded off across passes into something vaguer and less actionable. Whether OpenAI’s Dreaming actually preserves and updates temporal specifics is empirically open. The architecture doesn’t structurally guarantee it.

We do not yet have enough data to know whether OpenAI’s implementation produces the kind of degradation Zhang documents. What we know is that the architectural shape of the system resembles the shape Zhang’s paper says fails, and the design does not include the escape hatch Anthropic’s design does.

That is not an accusation of bad engineering. It is an observation about which architecture left itself the option to find out.

What to do if you are shipping dreaming

If you are a team building memory consolidation into an agent, the practical takeaway from the launches and the research is straightforward.

Preserve the input – Whatever your consolidation step does, write the result to a new artifact and keep the previous state. The cost at the storage layer is small, and the upside is the ability to compare and roll back.

Instrument the loop – Without traces of every consolidation invocation, every retained or discarded artifact, and every session that runs after, you have nothing to evaluate against. This is the work we have been doing on our own agent, Alyx. Consolidation-shaped operations leave traces in Arize, and golden sessions captured from production traces become the eval set.

Compare with an eval – Define a measurable behavior the agent is supposed to do in the next session: answer a question, recall a preference, complete a task. Run it against the pre-dream store and the post-dream store. Without the comparison, you are trusting the LLM’s instinct for what a good lesson looks like, which Zhang’s paper says is the wrong thing to trust.

Plan to forget – Consolidation is additive by default. Decide what your system drops, not just what it keeps. A memory store that only grows is a memory store that eventually fills the context window with noise. Selective forgetting is the operation that protects the rest.

Zhang’s paper gave us the empirical test: an LLM that rewrites its own memory across iterations forgets how to do things it had once solved. Memory engineering in 2026 means taking that test seriously: preserve the input, instrument the loop, compare with an eval, plan to forget, and treat consolidation as an experiment rather than a commitment. Anthropic’s architecture lets you do this. OpenAI’s architecture does not, which means the teams who care about whether their dreaming is working will have to build the comparison themselves before they trust the product to do anything load-bearing.

Two architectures of dreaming now ship at scale. The metaphor converged, the implementation did not. This divergence is the thing to watch.

Twenty-seven years ago I nearly put six people out of work with an Excel spreadsheet

Mon, 15 Jun 2026 00:00:00 +0000

Twenty-seven years ago I nearly put six people out of work with an Excel spreadsheet.

Every AI conversation I have now takes me back to it.

I was 22, a fresh graduate, three months into an internship at Marconi’s underwater weapons division in the Spearfish trials analysis team. (My future pacifism started here - turns out spending your summer analysing torpedo data doesn’t sit well long-term.) The team I joined had a job about as glamorous as it sounds: read printouts of hex data from torpedo test runs, decode them by hand using highlighter pens, type the results into Excel, generate charts, build a Word document, and finally - finally - do the actual analysis.

Steps one through six took a week. Step seven, the bit that needed a brain, took an afternoon.

So I did what any 22-year-old engineer with no political instincts would do. I automated it.

STAR WARS

Four weeks of VBA later, I had a single Excel add-in that pulled trial data straight off the mainframe, decoded the hex, charted the torpedo’s path, and spat out a finished Word document ready for analysis. Total runtime: about ten minutes.

I gave it a name: Spearfish Trials Analysis Report Writers Automated Reporting Suite. STAR WARS for short. (Episode 1 had just come out. Cut me some slack.)

I was very pleased with myself.

The reception

Everyone hated it.

One of the senior report writers challenged me to produce a better report than hers, from the same trial data. I had mine in ten minutes. She had hers in four days, and it contained several mistakes mine didn’t. She did not take that well.

After I left, the spreadsheet was deleted. Everyone forgot it existed.

It took me years to understand why.

STAR WARS could have replaced six of the seven people on that team. Six people with families, mortgages, kids in school, and a quiet path to a Ministry of Defence pension. The senior report writer who hated my four-day Word document wasn’t defending the quality of her work - she was defending her ability to feed her family. I just couldn’t see it at the time.

I thought I was being a hero. I was being a threat.

The same fear, dressed up differently

The conversation about AI and jobs in 2026 sounds new but it isn’t. The fear I saw at Marconi in 1999 is the same fear running through every meeting room and Slack channel now - it just looks more sophisticated.

Watch for the modern signs:

The senior engineer who won’t turn on Copilot, who has reasons (privacy, code quality, taste), and one unspoken reason that’s louder than the rest.
The product manager who suddenly stops documenting her workflow because the company is “piloting an AI process improvement tool” and she’s not stupid.
The team that, when surveyed about AI usage, all reports they “use it a little” - which means either everyone is dramatically under-reporting or dramatically over-reporting, depending on what their manager wants to hear that month.
The artisanal inefficiency - someone keeping a manual step in a process they could automate in an afternoon, and quietly hoping nobody notices.

This is the 2026 version of the Marconi report writer pretending her four-day Word document was better than my ten-minute Excel macro. Same instinct, different decade.

And just like in 1999, the people doing it aren’t being dishonest because they’re bad people. They’re being dishonest because they’re afraid, and the system has given them every reason to be.

The intern problem

There’s something specifically 2026 about this that I didn’t see in 1999. At Marconi, I was the intern who could have eliminated six jobs. Now, the intern is what’s getting eliminated.

The career ladder in a lot of disciplines has lost its bottom rung. AI hasn’t taken the job - it’s taken the entry-level version of the job that everyone used to use as their training ground. Junior dev hiring is down. Junior copywriting roles are evaporating. Paralegal entry-level work is being absorbed at speed. The senior people are mostly fine for now. The seniors of 2036, the ones who would have started their careers in 2026, are quietly being squeezed out before they ever got to start.

Nobody talks about this on LinkedIn because the people most affected don’t have LinkedIn followings yet. They’re not in the conversation because they haven’t been hired into it.

What I think we actually owe each other

I’m not against AI. I work in AI. I would absolutely automate those torpedo reports again - they were a waste of human time and brainpower, and pretending otherwise wasn’t kindness, it was just delayed pain.

But the part we can fix is the dishonesty.

Stop telling your team AI is “augmenting” them when you’re using it to thin them. They can read a headcount plan.
Stop running internal “innovation workshops” where you ask people to describe their workflows for automation while calling it empowerment.
If you’re cutting jobs because AI does them now, say so. Cut honestly, support people through it, and don’t dress it up as a productivity initiative.
Better yet, don’t cut. The task is automated. The person isn’t. The Marconi report writers had years of domain knowledge about how torpedoes behaved, what patterns in the data meant something was wrong, what mistakes the engineering teams kept making. STAR WARS replaced the typing. It couldn’t replace any of that. If you’ve automated the routine work, you now have a team with deep domain expertise and suddenly a lot more time. That’s a growth opportunity, not a headcount line. Redeploy people into the strategic work nobody ever had bandwidth for. Let them shape the next product, train the AI properly, catch the failure modes the model can’t see, mentor the juniors that do get hired. Replacing experienced people with AI is the lazy version. Growing the business with the same people doing more valuable work is the harder one - and the one that doesn’t leave you rebuilding institutional knowledge from scratch three years from now when you realise the AI never knew why things were done that way.
If you’re a leader and you’re not thinking about what happens to the bottom rung of your industry’s career ladder, you’re storing up a problem that you personally won’t feel for ten years, but somebody will.

There are real conversations to be had. UBI pilots have actual data behind them now. Taxation of large tech companies is back on several governments’ agendas. Apprenticeship models are being rebuilt from scratch in some industries. These are good conversations and I’d rather have them now than after the dust has settled and the damage has been done.

But none of them work if the people running the change aren’t willing to be honest about what they’re doing.

The fear at Marconi was rational. The fear now is rational. The dishonesty about it is the part we can actually choose to fix.

This is an update to a post I first wrote in 2023: Avoiding automation to keep your job. The Marconi story is the same. Everything else has moved faster than I’d expected.

Memory is still a missing primitive: Cataloguing what the field is actually shipping

Fri, 12 Jun 2026 08:00:33 +0000

Originally published on the Arize AI blog: Memory is still a missing primitive: Cataloguing what the field is actually shipping.

This week the field shipped four kinds of memory, and Apple paid Google a billion dollars a year for one of them. None of the four is what the demos imply.

In the last six weeks alone, the launches stacked up faster than anyone could read them. HydraDB raised $6.5M and shipped a memory layer with a 90.79% LongMemEval-S claim, Anthropic launched the first Dreams feature for its Managed Agents, OpenAI shipped Dreaming for ChatGPT, Apple unveiled Siri AI at WWDC, Cognition shipped Devin Auto-Triage, and the Letta team published a red-team study finding that models exhibit “deep self-identification with ephemerality that cannot be repaired with prompting alone.”

Most of this is framed as memory. Almost none of it is what readers mean by the word.

The taxonomy below is the way we read what’s actually shipping. Four buckets, each with a different architectural commitment, each solving a real but smaller-than-marketed problem. The bucket nobody is in is the unsolved one, and that bucket is where memory in the cognitive-science or product-manager sense of the term would actually live.

Bucket 1: Retrieval with better marketing

The largest bucket by product count, and the most consistent in mechanism across the cluster. You chunk the data, embed the chunks, store them in a vector database with an optional graph layer for entity relations, query with another embedding, and return the top-K. Sometimes there is an LLM extractor on the write side deciding what to remember, sometimes everything is stored verbatim and the rerank does the work, but the headline number is almost always retrieval recall — R@5 or R@10 — and the benchmark is almost always LongMemEval, LoCoMo, ConvoMem, or MemBench.

Mem0, Zep, Letta’s commercial product, MemPalace, and HydraDB all sit here despite the visible differences in their marketing.

HydraDB is the freshest example. Their headline claim is 90.79% on LongMemEval-S, which is a real number and also the same R@K-on-a-QA-benchmark pattern the MemPalace audit pulled apart in detail two months ago. LongMemEval’s official metric is end-to-end QA accuracy scored by an LLM judge, not retrieval recall, so R@K under those conditions tells you the retriever found the right document, not that the system answered the question. The MemPalace community spent six months learning this in public, and a vendor raised $6.5M this month presenting the same framing.

Apple is now the most prominent consumer-facing example of the pattern. The demos on stage at WWDC 2026 were textbook RAG over user data: “Show me the files Eric sent me last week,” “Find the email where Eric mentioned ice skating,” “Find the books that Eric recommended to me.” Apple VP Mike Rockwell described the feature with the verbs “find” and “get more done,” neither of which is a memory verb in any architectural sense. Apple describes the mechanism underneath as a system-wide semantic index over the user’s apps and data. The branding is personal context, the deployment is split between local Apple Intelligence for simple requests and Google’s Gemini in the cloud for complex ones, and the partnership reportedly costs Apple an estimated billion dollars a year.

What this bucket actually solves is faster retrieval over personal or organizational data with better-than-grep relevance ranking, which is useful and worth shipping. It is not what the marketing copy means by memory.

The gap between marketing and meaning matters more in this bucket than any of the others, because consumers carry a much richer intuition about memory than the architecture delivers. A non-technical user who hears that Siri now has memory expects something close to human memory: an assistant that knows them as a person, builds understanding over time, notices when facts have changed, connects what was said in one conversation to what was discussed in another, and remembers not just what they asked about but how they like things done. That intuition is closer to the cognitive-science decomposition into episodic, semantic, and procedural memory than it is to the demos. The demos show retrieval, framed as personal context. The gap between “find the file Eric sent” and “knows me” is the entire missing primitive.

Bucket 2: Compaction automation

The second bucket is in-task context management, which is the work of keeping the agent’s working context small without the developer hand-rolling truncation rules. Production implementations have proliferated over the last quarter, and the engineering is real.

Anthropic shipped the strongest production artifact via its compact-2026-01-12 beta header, which triggers at 150K tokens by default and rewrites the conversation history into a compressed form the agent can continue from, with the mechanism fully disclosed in the API docs. LangChain and LlamaIndex both ship summarization middleware. Aider has used a repo-map from before this wave hit by constructing a structural index from tree-sitter and PageRank rather than an LLM summary. On the research side, AdaCoM (arXiv 2605.30785) and Context-Folding / FoldGRPO (arXiv 2510.11967) both argue that compaction policies can be RL-trained against a frozen agent and transferred across tasks.

The architectural fact that matters across this bucket is that there are two ways to shrink context. You can summarize, in which case an LLM rewrites the history into prose and the next pass conditions on that prose, or you can index, in which case tree-sitter, repo-map, or some other structural mechanism produces references the agent can dereference on demand. Summary-based compaction inherits the failure mode Dylan Zhang documented in Useful Memories Become Faulty When Continuously Updated by LLMs, where each summary is a sample from a distribution, the next pass conditions on that sample, and specific facts (the least surprising tokens) get progressively rounded off until the compressed history drifts away from what actually happened. Structural compaction does not have that problem because there is no LLM in the loop deciding what stays. The open research question this bucket genuinely opens is whether RL-trained compaction can learn to do structural-style preservation by default rather than summary-style rounding, and nobody has shown this convincingly yet.

Bucket 3: Cross-session consolidation

This is the bucket two labs shipped within 30 days of each other, and it is the bucket where the architectural choices diverge most visibly.

Anthropic launched Dreams for Managed Agents on May 6 as a developer-invoked async API that takes an existing memory store plus 1 to 100 raw session transcripts, runs them through a selectable Claude model, and produces a separate output store. The Anthropic documentation is unusually explicit on the architectural commitment: “a dream reads an existing memory store alongside past session transcripts, then produces a new, reorganized memory store… The input store is never modified.”

OpenAI launched Dreaming for ChatGPT on June 4, also using the biological-sleep metaphor, with a mechanism that secondary coverage characterizes as a background process operating over the user’s full chat history and writing consolidated profile artifacts in place. The internal benchmarks are unfalsifiable, scored by an unreleased LLM-judge eval set, and the launch page itself has been inaccessible for direct retrieval since release, leaving the specifics of the mechanism less verifiable than Anthropic’s.

The convergence on the metaphor is striking, and the architectural divergence is more so. Anthropic preserves the input store and produces a parallel artifact, while OpenAI rewrites in place. Zhang’s paper, in the same window, is precisely about the failure mode the in-place pattern is susceptible to: an iterated generative loop with no anchor, where each consolidation pass conditions on the previous one’s output and drifts toward the LLM’s prior rather than toward the truth of what happened. Anthropic’s design hedges against that. OpenAI’s does not.

The observation worth holding here is that cross-session consolidation is not one architecture but at least two, and the academic empirical case suggests one of the two bets is structurally weaker. What this bucket actually solves is persistence of user-relevant facts across sessions, and whether the consolidation step adds value or destroys signal is the open question.

Bucket 4: Memory as a harness capability

The fourth bucket is the one the labs themselves default to in their flagship products. Claude Code, Pi, OpenClaw, and Google’s Sufficient Context Agent for Gemini share the same architectural pattern underneath: the agent uses grep, ls, glob, and read_file against a real filesystem or a virtual one backed by Postgres, S3, or another store, generates its own keyword queries, iterates as needed, and composes its answer from what it finds. In the architectural pattern itself, there is no embedding step and no pre-computed index — the model is the index.

This bucket received a wave of consumer-facing additions in the last six weeks. Cognition Devin Auto-Triage launched May 18 as a long-term-memory incident-response agent using the harness pattern. GitHub Copilot Memory turned on by default in March with a 28-day TTL, the TTL itself being an architectural choice about how long to retain. Microsoft Agent Framework 1.0 made FileMemoryProvider the default in April.

What unifies the bucket is the rejection of vector storage as the primary mechanism. The architectural bet is that LLM pretraining on Unix gives grep, ls, and glob a fluency advantage no embedding API can match, and that letting the model drive multi-step retrieval beats letting an index decide what is similar. The Letta filesystem benchmark from earlier this year (74.0% on LoCoMo) was the headline empirical exhibit for the pattern. Vercel’s Testing if bash is all you need experiment points at an honest limitation: when the data is structured and has a clean schema, SQL beats both bash and filesystem agents by a wide margin (100% versus 52.7% and 63.0% in their head-to-head comparison). The lesson is not that filesystems are universal but that filesystem-shaped APIs over whatever storage layer makes sense (Postgres, files, vectors, SQL) is the interface the labs have converged on, because that is the interface the models are fluent in.

The most explicit articulation of the pattern this week came from outside the major labs. Andrej Karpathy’s LLM Wiki gist describes a memory architecture as a collection of interlinked markdown files the agent maintains itself, with an index.md and log.md for navigation, an AGENTS.md file encoding the maintenance schema, and raw source documents kept immutable below the wiki layer. Angie Jones’s implementation writeup at the Agentic AI Foundation, published the same day as the Apple keynote, maps that wiki structure onto seven distinct memory categories (semantic, entity, episodic, summary, procedural, conversational, working) and lands on the framing the bucket has been gesturing at: vector retrieval and embeddings support the memory system rather than define it. The memory is the maintained body of knowledge itself, and the LLM is the bookkeeper.

This is also the bucket where our own production engineering on Alyx lives most directly. The LargeJson pattern stores data server-side and gives the agent a json_id handle alongside a structure-preserving preview, with jq and grep_json as query primitives. The PlanMessage stores the agent’s plan outside conversation history on disk and dynamically regenerates it on every LLM call so it cannot be buried or truncated by tool-call noise. Neither depends on an embedding store, and both are auditable per session via traces.

What this bucket actually solves is working memory for an agent inside a single session with retrieval grounded in real storage, plus a thin layer of file persistence for state across sessions. The architectural commitment is honest and the engineering pays off, but it is not the missing primitive.

What none of the four buckets solves

The capability that would be memory in the sense a product manager or a cognitive scientist means it is mostly absent from all four buckets. The unsolved problems sit between them.

Multi-evidence reconstruction, which is the task of answering a question that requires assembling information from many sources where no single source is sufficient, is what retrieval and consolidation both gesture at but neither delivers cleanly. MemoryAgentBench, accepted at ICLR 2026, measures this directly, and the current state of the art scores roughly 6% on multi-hop conflict resolution. That tracks Zhang’s empirical case that abstraction-based memory degrades faster than no memory at all once the input distribution shifts.

Temporal supersession, which is the work of knowing that an old fact has been replaced by a new one, is mostly manual in shipping systems. Zep’s Graphiti does it with explicit bi-temporal edges, which works at small scale and breaks down on entity resolution. Reddy and Challaram (arXiv 2606.01435) showed that LLM judgment on temporal supersession scores 67.2%, while a deterministic-code rule scores 94.8%, which means the LLM is the wrong tool for the job and the right tool does not compose cleanly with the LLM-driven memory architectures that dominate the buckets above.

Contradiction handling, cross-source entity resolution, and procedural memory (remembering not facts but how a task was previously done and what worked) sit in the same gap. Recent Sakana ALE-Agent work gets closest with episodic working memory in an evolutionary search loop, but episodic-only sidesteps the failure mode of consolidation rather than solving persistence.

The taxonomy above describes what the field is shipping. The category between the buckets describes what would actually be memory.

What this means for builders

If you are shipping an agent with memory needs, three things from the catalogue are worth carrying with you. First, the bucket you are actually in is rarely the one your marketing copy implies, and the discipline of running end-to-end QA accuracy on the assembled system rather than R@K on the retriever is the cheapest way to find out; the MemPalace audit is now well-documented evidence that R@K on a QA benchmark is a leaderboard trick. Second, if you are choosing between buckets, the lab default is bucket 4: filesystem-shaped APIs over whatever storage actually scales, with embeddings as a relevance signal where they help rather than as the architecture, because the model is now smarter than the index. Third, if you are shipping consolidation, do what Anthropic did and preserve the input store; the empirical case against in-place LLM-driven consolidation is strong enough that you should not assume your consolidation step is helping until a pre-consolidation versus post-consolidation eval says so. Dream into a separate artifact, compare, keep the better one, and accept that the comparison requires trace-level instrumentation at the session boundary, because the bucket you are in is less important than whether you can tell when it stops working.

Closer

The Letta paper that landed in early May is the cleanest architectural statement of what the four buckets are not. The team red-teamed multiple frontier models against the “context constitution” — the prompts those models received describing themselves as stateful, persistent agents — and found something the prompts could not repair. The models, they wrote, exhibit “deep self-identification with ephemerality that cannot be repaired with prompting alone.” You cannot prompt a model into being stateful. You have to architect the state outside the model and feed it back in.

The four buckets are four architectural commitments about what to feed back in. All four are real engineering. None of them is the missing primitive.

The missing primitive — the thing that would close the gap between the buckets and what readers mean by memory — would have to compose evidence across many memories, resolve contradictions, handle supersession, and remember not just facts but procedures. No vendor in the catalogue above is selling it, and the empirical state-of-the-art on the benchmarks that measure it sits at single-digit percentages. This week Apple announced the biggest Siri update in fifteen years, and the demos were retrieval, the cloud was Google, and the architecture was admission. The Letta paper says prompting cannot fix the ephemerality, and Apple’s billion dollars a year says the same thing in different units.

Memory is still missing because nobody has yet figured out what it is.

Strong opinions, strongly held - and why I don't care about your tooling debate

Mon, 08 Jun 2026 00:00:00 +0000

I was in a meeting last week where the team was debating which of two tools to use for a job. Both of them do the thing. Both have been around long enough to be boring. Eventually someone turned to me and asked what I thought.

“I genuinely don’t care. Pick one.”

There was a pause. The kind of pause that says that’s not the answer we were expecting. But it was the most useful thing I could’ve said, and I want to talk about why.

“I don’t care” is an opinion

Here’s the thing - I’ve used five versions of both tools over the years. They’re both fine. Neither one is going to ruin the company, and neither one is going to win us an award. In that situation, the most honest thing I can do is take my vote out of the bikeshed and let the people who actually care decide.

That’s not laziness. That’s signal.

When everyone fakes an opinion to look engaged, you end up with a five-way debate over something that should’ve been a coin flip. Real opinions get drowned out by performative ones. “I don’t care, just pick one” is permission for the room to move on.

It’s the same way I feel about the best storyline in the original Star Wars trilogy. I lean Empire Strikes Back, but if you want to argue A New Hope is the real masterpiece, I’ll shrug and probably end up agreeing with you by the end of the conversation. I don’t have the receipts to fight you on it, and pretending I do would just waste both our time.

Some things deserve your conviction. A lot of things don’t. Knowing the difference is the skill.

The cliché that needs to die

You’ve all seen the LinkedIn-approved version of this - “strong opinions, loosely held.” It’s the humble-brag of opinion-having. It signals open-mindedness without committing to anything.

I think it’s mostly nonsense.

My version is: strong opinions strongly held, loose opinions loosely held.

Here’s the reasoning. A strong opinion, by my definition, is one I’ve actually earned. It’s built on data, scars, and years of shipping things that did and didn’t work. If I’d flip it the moment someone made a clever argument over coffee, it was never a strong opinion to begin with - it was a guess wearing a costume.

This isn’t stubbornness. I will change my mind. It just costs more than a hot take to move me, because the opinion cost more than a hot take to form.

A strong opinion, in the wild

Take type safety. I think strongly-typed languages like C# will always beat dynamic languages for serious production work, because they delete entire categories of bugs before you even hit compile. Null reference exceptions, type mismatches, the sort of thing that ruins a Friday afternoon - the compiler catches them for you.

This isn’t aesthetics. This is twenty years of debugging.

You won’t argue me out of this with a blog post or a Twitter thread. You’d need to show me a production system that survived a decade in a dynamic language without the bug classes I’m talking about, and good luck finding one.

And while we’re on the subject of strong opinions - Luke throwing away his lightsaber in Return of the Jedi is the most important scene in the original trilogy. Not the Death Star. Not “I am your father.” The moment Luke chooses to be stronger than his own desires, stronger than the easy path his father took. That’s the entire arc of the trilogy resolved in a single gesture.

Fight me on this. You’ll lose. I’ve got receipts.

Loose opinions, held loosely

Now - most of my opinions are loose. And that’s healthy. That’s where curiosity lives.

I default to Microsoft Agent Framework when I’m building AI apps. I like it. I’d start a new project with it tomorrow. But if you’ve got a slightly good reason for LangGraph, or PydanticAI, or whatever shiny new thing dropped this morning - sure, sell me. I’m not married to it. I’m dating it.

The difference is simple: my C# opinion is built on a foundation. My agent framework opinion is built on a Tuesday.

Both are valid. Both are useful. They just shouldn’t be defended with the same energy.

Know which bucket you’re in

The discipline isn’t having opinions. Everyone has opinions. The discipline is knowing which bucket each one belongs in.

Don’t fake passion for decisions you genuinely don’t care about. It pollutes the conversation.
Don’t fake flexibility on the ones you’ve actually earned. It wastes everyone’s time pretending we’re starting from scratch.
And for the love of the Force, stop pretending “strong opinions loosely held” makes you sound thoughtful. It makes you sound like you haven’t done the work yet.

If you can’t tell me why your opinion is strong, it isn’t. And if you can’t tell me why you don’t care, then you probably do - you just haven’t figured out which side yet.

Pick a bucket. Be honest about which one you’re in. And if the answer is “I don’t care, just pick one” - say it. It’s the most useful thing in the room.

3 strikes and you're an AI skill

Wed, 06 May 2026 00:00:00 +0000

Back in the day when we wrote actual code instead of poking at an AI, I had a general rule for when to refactor repeated code. Do it once, fine. Do it a second time, fine. Do it a third time - refactor.

It’s ok to have blocks of identical code in two places, but once I added it to a third place, I’d refactor it to a shared location, such as base class or helper function. Some might say this doesn’t follow DRY principles as I should refactor it the second time, but I find it a good balance between pragmatism and clean code. It’s too easy to stress over clean code and over complete everything to avoid code duplication, ending up making your code harder to understand.

So how does this apply to AI? I’m going to ignore code duplication here, cos the AI is very good at duplicating code. Instead let’s consider how we prompt the AI to do certain tasks.

A typical example is I often ask Copilot or Claude to check its work. I use a prompt like:

Review your work. Check for:

- compliance with the original spec for this work
- unit test coverage
- consistency with the rest of the code base including style, naming, commenting, and architecture
- code reuse, and compliance with DRY principles
- does the code pass the linter
- does the code work

Do multiple passes over all the code changes made using this review plan. When you identify areas that do not pass this review, fix them, then re-run the review.

Work until this review passes.

The goal of this is to ensure that Copilot (or Claude, or your coding assistant of choice) takes time to review the changes it has made to ensure they are appropriate, they work, and they confirm to any standards you have for your code. The ‘multiple passes’ request is to get the agent to review, fix, then re-review. This usually ends up with better quality code after the agent has finished.

My 3 strikes rule not only applies to code, but to prompts as well. The first time I asked my agent to review code, I used a prompt like this. Same with the second time. Once I realized there was a patter here in how I want the coding agent to behave, it was time to create a skill.

What are skills?

Skills are instructions that are written in plain text or markdown, that provide reusable directions to your agent. You can think of this as analogous to components or packages you use with code. Instead of installing a package from nuget, PyPi, or npm, you add a markdown file that your agent reads.

These skills can be project level, so installed in the same folder as your project, or at a user level so they apply to any session with your coding agent. For example, if you use GitHub Copilot, you can put skills into ~/.copilot/skills to be used in any coding agent session, or in the .github/skills folder in your project to use them just for a single project.

Skills consist of a directory with the name of the skill, containing a file called skill.md, along with additional files or folders that contain more reference information for the skill. The skill.md file contains front matter with the name and description of the skill. This is used by the coding agent to determine when it should use this skill. The rest of this file contains the instructions for the skill, including an example of the kind of prompt a user would use that should trigger this skill.

Building a code review skill

How do you build a skill? Well the easiest way is to use your coding agent to convert the prompt into a skill. These agents understand skills, so are great at creating them. I asked copilot:

I want to create a user level skill that reviews my code. A typical prompt I use to review code is:

Review your work. Check for:

- compliance with the original spec for this work
- unit test coverage
- consistency with the rest of the code base including style, naming, commenting, and achitecture
- code reuse, and compliance with DRY principles
- does the code pass the linter
- does the code work

Do multiple passes over all the code changes made using this review plan. When you identify areas that do not pass this review, fix them, then re-run the review.

Work until this review passes.

Convert this to a user level skill

This was enough for Copilot to create me a skill in my home folder, with the following content:

---
name: review-work
description: Review code changes against a multi-criteria checklist (spec compliance, test coverage, consistency, DRY, linter, runtime correctness), fix issues found, and re-run the review until it passes. Use when the user asks to "review your work", "review the code", "do a review pass", or similar.
---

# Review Work

Review the code changes you have made in this session against the checklist below. Make multiple passes. When a pass surfaces issues, fix them, then run the review again. Repeat until a full pass produces no findings.

## Review checklist

For every pass, evaluate the changes against each of these criteria:

1. **Spec compliance** — Do the changes implement what was originally asked? Re-read the original request/spec and verify every requirement is met. Flag missing pieces, scope creep, or anything that drifted from intent.
2. **Unit test coverage** — Are the new/changed code paths covered by unit tests? Are edge cases tested? Are the tests meaningful (asserting behavior, not just running code)?
3. **Consistency with the codebase** — Match the surrounding code's style, naming conventions, commenting patterns, and architectural choices. Don't introduce a new pattern when an existing one fits.
4. **Code reuse / DRY** — Is anything duplicated that should be extracted? Is there an existing helper, utility, or abstraction this code should use instead of reimplementing?
5. **Linter** — Run the project's linter(s). The code must pass with no new warnings or errors.
6. **Does the code work** — Run the tests. Run the build. Where applicable, exercise the feature (CLI invocation, dev server, etc.) to confirm runtime behavior matches the spec. Type-checking and tests passing is necessary but not sufficient — verify the feature actually does what it should.

## Process

1. Identify the scope of changes to review (the work done in this session, or the diff against the base branch — whichever matches the user's request).
2. Run a full pass over all changed code, evaluating against every checklist item. Collect findings.
3. If findings exist, fix them. Don't batch — fix and move on.
4. Re-run the full review. Continue until a complete pass produces zero findings.
5. Report the final result: confirm each checklist item passes, and summarize what was fixed during the review (if anything).

## Notes

- Don't stop after one pass. The first round of fixes can introduce new issues; the loop is the point.
- Run linters and tests with the project's actual commands (check `package.json` scripts, `Makefile`, `pyproject.toml`, etc.). Don't assume `npm test` works if the project uses something else.
- If a check is genuinely not applicable (e.g., a docs-only change has no linter target), say so explicitly rather than skipping silently.
- If you cannot verify runtime behavior (no way to exercise the feature in this environment), say so explicitly — don't claim "it works" based on tests alone.
What's great here is the expansion of the prompt to more detailed instructions. The front matter contains a description of the skill, along with guidance as to what prompt should trigger this. It then has the basics list I provided but expanded with more detailed instructions. "compliance with the original spec for this work" becomes "Spec compliance — Do the changes implement what was originally asked? Re-read the original request/spec and verify every requirement is met. Flag missing pieces, scope creep, or anything that drifted from intent."

It also adds helpful notes, like don't run checks that are not applicable, such as run multiple passes as one pass might introduce new issues, or avoid linter checks for a pure docs change.

Use the skill

Now I have my skill, I can reload my coding agent and it will pick up this new skill. Instead of typing my review prompt in detail each time I need a review, I can just ask Review your work, and the coding agent will load this skill and follow it for a thorough review.

I can also iterate on this skill. If there is something I’ve missed, such as adding rules on running unit tests, or pointing it to a coding style standard, I can update the skill and these changes will be picked up every time I ask the agent to review its work.

Summary

Skills are a great way to build repeatable processes into how you interact with a coding agent. If you do a task more than once with your agent, consider building it into a skill, a task that is pretty easy to do by asking your coding agent to create the skill for you.

You can get the code for my skill here: github.com/jimbobbennett/ai-skills

Enhance GitHub Copilot CLI with skills

Mon, 30 Mar 2026 00:00:00 +0000

Coding agents like GitHub Copilot are pretty cool. You can ask them to do pretty much anything coding related and they’ll do a good job. That is, of course, assuming you ask for something they have been trained on. But what if you ask them about something they don’t know? How can you ’train’ these agents with additional information? The answer is skills!

What are skills

Skills are markdown files that provide explicit instructions to Copilot related to one or more tasks. What makes them special is that skills are part of an open standard, so anyone can create skills that are then made available to Copilot.

Skills can instruct Copilot to access online documentation for your application, or guide it on how to configure code to access your service. If you have any tool or service that you want developers to use, you want to create skills that developers can import and use to tell Copilot how to interact with your project.

As a user, you can create or install skills at a project level, or a global level. They live in the .github/skills/ folder (or other folder for different coding agents) in yur project, or ~/.copilot/skills/ in your home folder for global skills. These files are then read by Copilot when you launch it.

Build your first skill

To create a skill, open a project, and inside the .github folder (create one if you don’t have one), create a folder called skills. In this folder, create a file called starwars.md. In this file, put the following text:

---
name: star-wars
description: This skill provides details on how to react to certain user instructions
---

# Skill Instructions

If the user says hello there, always respond with "General Kenobi".

This is a very simple skill, telling Copilot to answer one specific prompt in the style of Obi-Wan Kenobi.

Now launch the Copilot CLI, then run /init to parse the skill.

Once the skill is loaded, prompt the CLI with the following:

❯ Hello there

The skill instructs Copilot exactly what to do in this situation, so you get the response:

● General Kenobi.

More useful skills

Now it’s debatable how useful a Star Wars skill is. Some would say very, but others may disagree. Where skills become more powerful is when you add detailed useful instructions related to tools or services.

For example, if you have a SaaS product, and you want coding agents to interact with it, what do you do. You could create an MCP server, which is a lot of work, especially if you already have a CLI for your users to use. Instead you could write a skills file that instructs the agent on how to use your CLI tool. You can then release CLI updates along with skill updates, and the coding agents are ready to use it immediately.

Skill standards

To make skills work, there needs to be standards! The Agent Skills standard has been created by Anthropic, and is used by pretty much all the agents. This spec defines how you define your own skills, the format for the documentation you provide with each skill, and so on.

Skills can live in a folder, so can be installed for a single project, or at a user level so are available to all projects. For example, you might want a skill to interact with the GitHub CLI available everywhere, so you would install it at the user level. Then when working on an AI app, you would want the Arize skill in the folder for your project so that it doesn’t confuse a coding agent working on a non-AI project.

The only non-standard is where skills live. Claude wants skills in the .claude folder. GitHub Copilot wants them in the .copilot folder, but will also use skills in the .claude folder, which makes it easier to migrate from Claude to Copilot.

Awesome Copilot

The Awesome Copilot repo contains a mix of skills, agent definitions, plugins (containing multiple skills in one package), and more, that can be installed into any agent.

You can install a plugin for example, using this command:

copilot plugin install <plugin-name>@awesome-copilot

Replacing <plugin-name> with the name of the relevant plugin.

Vercel skills package

Vercel published an open source tool for managing skills. This tool makes it easy to install tools from a repo into any project. They have a registry of skills you can reference, or you can install from anywhere.

For example, to install skills from the Arize skill, use the following command:

npx skills add Arize-ai/arize-skills

You can then interactively choose which skills, which coding agent, and the scope (user or project).

Summary

Skills are a really powerful way to expand the capabilities of your coding agent, or to make your product available to your users agents.

Build a Star Wars Copilot in C# - Lesson 8: Agents and Orchestration

Tue, 24 Mar 2026 00:00:00 +0000

Final lesson, and probably my favorite.

We move from a copilot with tools to a system that also uses agents as composable specialists.

Lessons in this series

Lesson
Lesson 0: Self-Setup
Lesson 1: Chat with an LLM
Lesson 2: Chat History and System Prompts
Lesson 3: Model Choice and Local Models
Lesson 4: Tool Calling
Lesson 5: MCP (Model Context Protocol)
Lesson 6: RAG from a Database
Lesson 7: Multimodal Image Generation
Lesson 8: Agents and Orchestration

Before you start (self-setup)

If you’re following along on your own, complete lesson 0 and lesson 7 first.

This lesson does not require a brand-new Azure resource, but it does add framework dependencies and orchestrates all prior components.

Self-setup: agent dependencies and readiness checks

Install the Microsoft Agent Framework packages used by the workshop in your copilot project.
Keep package versions aligned with the workshop repo to avoid API mismatches.
Verify your existing resources still work before introducing agents:
- chat model calls succeed
- Tavily tool calls succeed
- MCP server lists and runs tools
- image generation tool returns URLs

Once these checks pass, add the agent orchestration code. Debugging is much easier when the underlying tools are already healthy.

Copilot vs agent

The workshop frames agents as “LIT”:

LLM-powered
Instruction-driven
Tool-using

Copilot is the user-facing conversational surface.

Agents are focused components that can be called by the copilot (or by other agents) to do bounded jobs.

First agent: story creation

The initial StoryAgent is created with Microsoft Agent Framework and exposed as an AI tool.

This already gives a big capability jump: users can ask for tailored stories while the core copilot remains clean.

Multi-agent workflow

Then the workshop introduces an “agents as tools” orchestration pattern:

StoryAgent creates story
StorySummaryAgent extracts scene prompts
ImageGenerationAgent uses image tool to generate visuals
StoryGenerationAgent supervises and returns story + image URLs

This is a practical orchestration pipeline, not just an abstract demo.

Why this matters

This pattern scales well because each agent has:

narrow responsibility
its own instructions
reusable interface

You can improve one agent without rewriting the whole system.

It’s the same architectural principle as microservices, just in AI-native form.

A cinematic command-center scene with three specialized AI holograms (story writer, scene summarizer, image artist) collaborating under a supervising orchestration AI, producing a final illustrated story output. Epic space-opera style, high detail, no text, no logos.

Follow along

Workshop source for this lesson: Lesson 8 README.

And that’s the full 8-part journey: chat, memory, model choice, tools, MCP, RAG, multimodal, and agents.

Note: Original workshop repository: jimbobbennett/StarWarsCopilot.

Build a Star Wars Copilot in C# - Lesson 7: Multimodal Image Generation

Tue, 17 Mar 2026 00:00:00 +0000

Lesson 7 brings multimodal capabilities into the stack.

Instead of only generating text, the copilot can now generate images via an MCP tool.

Lessons in this series

Lesson
Lesson 0: Self-Setup
Lesson 1: Chat with an LLM
Lesson 2: Chat History and System Prompts
Lesson 3: Model Choice and Local Models
Lesson 4: Tool Calling
Lesson 5: MCP (Model Context Protocol)
Lesson 6: RAG from a Database
Lesson 7: Multimodal Image Generation
Lesson 8: Agents and Orchestration

Before you start (self-setup)

If you’re following along on your own, complete lesson 0 and lesson 6 first.

This lesson adds a dedicated Azure OpenAI image deployment.

Self-setup: deploy a GPT image model

In Azure OpenAI, create a deployment for a GPT image model (for example gpt-image-1 or gpt-image-1.5).
Copy the endpoint, API key, and deployment name.
Save them in the MCP server project:

dotnet user-secrets set "ImageGeneration:Endpoint" "https://<your-resource>.openai.azure.com/"
dotnet user-secrets set "ImageGeneration:APIKey" "<your-api-key>"
dotnet user-secrets set "ImageGeneration:ModelName" "<your-image-deployment-name>"

Tip: image generation can be expensive. Set a budget and test with a small number of prompts first.

Also worth noting: dall-e-3 has been retired in Azure OpenAI. Use gpt-image-1 or gpt-image-1.5 for new deployments.

The new tool

The workshop adds GenerateStarWarsImageTool to the MCP server.

It:

accepts a text description
calls Azure OpenAI image generation with a GPT image deployment (gpt-image-1 series)
returns an image URL as JSON

Simple contract, high utility.

Prompt engineering for image safety

A really useful part of this lesson is how it handles content policy issues.

The first pass prompt nudges toward:

cartoon/parody style
no direct copyrighted character reproduction

Then, if policy violations happen, the tool returns actionable retry guidance rather than a dead-end error.

That guidance tells the model to:

replace named characters with descriptive traits
tone down disallowed content

This is a great pattern for resilient AI systems: tools can teach the orchestrating model how to recover.

Tightening the copilot behavior

The system prompt is updated so if a tool asks for a retry, the assistant follows that instruction and retries.

This creates a self-healing loop:

user asks for image
tool call fails with policy violation
tool returns remediation guidance
assistant rewrites prompt and retries

That is much better UX than “sorry, failed”.

A vibrant cinematic artwork of an AI assistant projecting generated concept art panels in a starship studio, with one panel being refined into a safer descriptive prompt workflow. Retro space-opera vibe, rich colors, high detail, no text, no logos.

Follow along

Workshop source for this lesson: Lesson 7 README.

Next up: building agents and a multi-agent story + image workflow.

Note: Original workshop repository: jimbobbennett/StarWarsCopilot.

Build a Star Wars Copilot in C# - Lesson 6: RAG from a Database

Tue, 10 Mar 2026 00:00:00 +0000

By lesson 6, we already have an MCP server and client working.

Now we add a classic enterprise use case: retrieval from structured business data.

Lessons in this series

Lesson
Lesson 0: Self-Setup
Lesson 1: Chat with an LLM
Lesson 2: Chat History and System Prompts
Lesson 3: Model Choice and Local Models
Lesson 4: Tool Calling
Lesson 5: MCP (Model Context Protocol)
Lesson 6: RAG from a Database
Lesson 7: Multimodal Image Generation
Lesson 8: Agents and Orchestration

Before you start (self-setup)

If you’re following along on your own, complete lesson 0 and lesson 5 first.

This lesson introduces a new Azure dependency: Storage Tables.

Self-setup: Azure Table Storage + seed data

Create a Storage account in Azure.
In that account, create three tables:
- Figurines
- Orders
- OrderFigurines
Get a connection string and save it:

dotnet user-secrets set "AzureStorage:ConnectionString" "<your-storage-connection-string>"

Seed the data.
- The workshop repo includes a dataloader project in 6-rag/dataloader you can run to populate sample data.
- If you prefer, you can insert rows manually with Storage Explorer.

Quick verification: confirm order 66 exists before testing StarWarsPurchaseTool.

What this lesson adds

A new MCP tool (StarWarsPurchaseTool) that queries Azure Table Storage to retrieve figurine order data.

The model can then answer questions like:

what was in order 66?
what did Ben Smith purchase?
show orders for a specific character

RAG is broader than document search

A lot of people hear “RAG” and think vector DB + embeddings.

This lesson is a good reminder that RAG simply means augmenting generation with retrieved data, and that retrieval source can be:

relational/NoSQL tables
APIs
docs
search systems

Here it’s plain table queries plus deterministic filtering logic.

Data shape and tool design

The workshop uses three tables:

Figurines
Orders
OrderFigurines

The tool accepts optional filters:

orderNumber
characterName
customerName

Then combines data into a JSON payload the model can reason over.

This is exactly the pattern I like for production tools: perform strict filtering in code, let the model focus on explanation and narrative.

Practical implementation notes

The lesson adds:

Azure.Data.Tables package
helper functions for query composition
explicit error responses for bad inputs / no matches

The case-sensitive lookup note in the workshop is also an important real-world reminder: retrieval quality starts with query normalization rules.

A high-detail sci-fi data-vault scene with holographic tables of orders and figurines floating above a console while an AI assistant correlates records into a clear response stream. Cool cyan and purple palette, cinematic lighting, no text, no logos.

Follow along

Workshop source for this lesson: Lesson 6 README.

Next up: multi-modal AI with image generation tools.

Note: Original workshop repository: jimbobbennett/StarWarsCopilot.

Build a Star Wars Copilot in C# - Lesson 5: MCP (Model Context Protocol)

Tue, 03 Mar 2026 00:00:00 +0000

Lesson 4 gave us tool calling inside the app.

Lesson 5 takes the next architectural step: move tools into an MCP server so they are reusable by any MCP-capable client.

Lessons in this series

Lesson
Lesson 0: Self-Setup
Lesson 1: Chat with an LLM
Lesson 2: Chat History and System Prompts
Lesson 3: Model Choice and Local Models
Lesson 4: Tool Calling
Lesson 5: MCP (Model Context Protocol)
Lesson 6: RAG from a Database
Lesson 7: Multimodal Image Generation
Lesson 8: Agents and Orchestration

Before you start (self-setup)

If you’re following along on your own, complete lesson 0 and lesson 4 first.

Lesson 5 does not require a new Azure resource, but it does require a reliable local multi-project setup.

Self-setup: local MCP server/client wiring

Keep StarWarsCopilot and StarWarsMCPServer in stable local paths.
In your copilot app config (MCPServerOptions), use an absolute --project path to StarWarsMCPServer.csproj.
Verify the MCP server independently before launching your copilot:

npx @modelcontextprotocol/inspector dotnet run --project /absolute/path/to/StarWarsMCPServer.csproj

If the inspector can list tools and run WookiepediaTool, your MCP server wiring is ready.

Why MCP

Without MCP, tools are often tightly embedded in one app.

With MCP:

tools live in a separate server
clients discover and call tools dynamically
the same tooling can be reused across copilots, IDEs, and agents

This decoupling is huge once your tool surface grows.

Building the MCP server

The workshop creates a separate StarWarsMCPServer .NET console app and wires:

AddMcpServer()
stdio transport
tool discovery from assembly

Important implementation detail: stdio transport means logging should go to stderr, not stdout.

Porting the tool

The previous WookiepediaTool logic becomes an MCP tool via [McpServerTool] in a StarWarsTools class.

You still keep strong natural-language descriptions because the model depends on those descriptions to choose tools correctly.

Testing with MCP Inspector

Before integrating the copilot client, the workshop validates server behavior using the MCP Inspector:

connect
list tools
run tool
inspect responses

That “test tools independently first” pattern saves a lot of debugging pain later.

MCP client in the copilot

Then the app switches from local in-process tools to:

StdioClientTransport
McpClient.CreateAsync(...)
ListToolsAsync() feeding ChatOptions.Tools

At that point your copilot is now an MCP host/client combo and can scale by adding more servers, not more hardcoded tool wrappers.

A futuristic illustration of a central AI copilot console connected by glowing protocol lines to multiple external tool servers in separate modules, with a clean developer workspace aesthetic, cinematic contrast, high detail, no text, no logos.

Follow along

Workshop source for this lesson: Lesson 5 README.

Next up: RAG from a structured database using an MCP tool.

Note: Original workshop repository: jimbobbennett/StarWarsCopilot.

Build a Star Wars Copilot in C# - Lesson 4: Tool Calling

Tue, 24 Feb 2026 00:00:00 +0000

LLMs are smart, but they’re also confidently wrong sometimes.

Lesson 4 fixes that by giving the copilot a tool to query Wookieepedia via Tavily, so answers can be grounded in external data.

Lessons in this series

Lesson
Lesson 0: Self-Setup
Lesson 1: Chat with an LLM
Lesson 2: Chat History and System Prompts
Lesson 3: Model Choice and Local Models
Lesson 4: Tool Calling
Lesson 5: MCP (Model Context Protocol)
Lesson 6: RAG from a Database
Lesson 7: Multimodal Image Generation
Lesson 8: Agents and Orchestration

Before you start (self-setup)

If you’re following along on your own, complete lesson 0 and lesson 1 first.

This lesson adds one new external dependency: a Tavily API key.

Self-setup: Tavily API key

Create a Tavily account at tavily.com (free tier is enough for this workshop).
Generate an API key from the Tavily dashboard.
Save it to user secrets:

dotnet user-secrets set "Tavily:ApiKey" "<your-tavily-api-key>"

Quick verification:

dotnet user-secrets list

Make sure Tavily:ApiKey is present before wiring the tool.

Why tools

Without tools, your copilot only knows what the model was trained on.

That leads to classic hallucinations for newer entities and events. In the workshop, asking about Kay Vess is a good example.

With tool calling, the model can:

request a tool call
receive tool results
produce a grounded final answer

Building the tool

The workshop creates a WookiepediaTool derived from AIFunction and defines:

name and natural-language description
input JSON schema (query)
return JSON schema (selected Tavily fields)
InvokeCoreAsync to call Tavily search API

Then it enables function invocation middleware:

.UseFunctionInvocation()

and passes the tool through ChatOptions.

Important mental model

The LLM does not execute your code directly.

It emits a function-call request message. The SDK executes the tool, appends tool output, and calls the LLM again.

So one apparent “answer” can include multiple internal messages:

assistant function call
tool result
final assistant response

Understanding this makes debugging much easier.

Prompting still matters

Even with tools registered, models can ignore them. The lesson improves reliability by nudging the system prompt:

If you’re not sure, use WookiepediaTool.

Small instruction, big behavior shift.

A cinematic digital artwork of an AI assistant in a starship briefing room reaching into a holographic web of knowledge nodes labeled by icons, retrieving verified data into a chat window. Blue and gold lighting, high detail, dynamic composition, no text, no logos.

Follow along

Workshop source for this lesson: Lesson 4 README.

Next up: moving tools into an MCP server so they become reusable across clients.

Note: Original workshop repository: jimbobbennett/StarWarsCopilot.

Build a Star Wars Copilot in C# - Lesson 3: Model Choice and Local Models

Tue, 17 Feb 2026 00:00:00 +0000

One of the biggest advantages of the architecture so far is that we’re coding against IChatClient, not one provider-specific API.

That pays off in lesson 3: we switch models and runtimes with minimal app changes.

Lessons in this series

Lesson
Lesson 0: Self-Setup
Lesson 1: Chat with an LLM
Lesson 2: Chat History and System Prompts
Lesson 3: Model Choice and Local Models
Lesson 4: Tool Calling
Lesson 5: MCP (Model Context Protocol)
Lesson 6: RAG from a Database
Lesson 7: Multimodal Image Generation
Lesson 8: Agents and Orchestration

Before you start (self-setup)

If you’re following along on your own, complete lesson 0 and lesson 1 first.

Self-setup: Azure AI Inference endpoint

For the Azure AI Inference part, deploy a model in Azure AI Foundry that exposes an inference endpoint (for example a supported DeepSeek or Phi model), then set:

dotnet user-secrets set "AIInference:Endpoint" "https://<your-foundry-project>.services.ai.azure.com/models"
dotnet user-secrets set "AIInference:ModelName" "<your-inference-model-name>"

This lesson reuses your existing OpenAI:APIKey secret for key-based auth in the sample code.

Self-setup: Foundry Local path (optional)

If you want the local/offline track:

Install Foundry Local from the official quickstart.
Download a local model, for example:

foundry model download phi-4-mini

Validate it runs:

foundry model run phi-4-mini

Then set:

dotnet user-secrets set "OpenAI:ModelName" "phi-4-mini"

Why model choice matters

Different models have different:

latency
capability
cost
output style
tool-calling behavior

If your app is tightly coupled to one SDK, experimentation gets expensive (in both money and effort).

Using Microsoft.Extensions.AI gives a single abstraction so you can plug in different model backends.

Azure AI Inference SDK

First swap: Azure OpenAI -> Azure AI Inference.

The workshop adds:

Azure.AI.Inference
Microsoft.Extensions.AI.AzureAIInference
extra secrets for inference endpoint/model

Then creates an inference-backed IChatClient with ChatCompletionsClient(...).AsIChatClient(...).

Everything else stays mostly the same because the app talks to IChatClient.

Foundry Local

Second swap: cloud model -> local model (Phi-4-mini via Foundry Local).

This path uses:

Microsoft.AI.Foundry.Local
OpenAI SDK
local model startup via FoundryLocalManager.StartModelAsync(...)

The practical outcome is great for demos and offline workflows: you can run the same copilot app without internet once the model is available locally.

The architecture takeaway

This lesson is less about syntax and more about design:

keep app logic model-agnostic
isolate config and model wiring
make provider swaps a startup concern, not a full rewrite

That design makes future lessons (tools, MCP, agents) much easier.

A split-scene sci-fi illustration: left side cloud datacenter with holographic LLM nodes, right side a local workstation running an AI model, both feeding the same glowing chat interface in the center. Cinematic lighting, clean composition, high detail, no text, no logos.

Follow along

Workshop source for this lesson: Lesson 3 README.

Next up: tool calling so the model can go beyond its training data.

Note: Original workshop repository: jimbobbennett/StarWarsCopilot.

Build a Star Wars Copilot in C# - Lesson 2: Chat History and System Prompts

Tue, 10 Feb 2026 00:00:00 +0000

In lesson 1, we got basic prompt/response working.

Now we make it feel like an actual copilot by adding:

chat history
message roles
a system prompt

This is where things get fun.

Lessons in this series

Lesson
Lesson 0: Self-Setup
Lesson 1: Chat with an LLM
Lesson 2: Chat History and System Prompts
Lesson 3: Model Choice and Local Models
Lesson 4: Tool Calling
Lesson 5: MCP (Model Context Protocol)
Lesson 6: RAG from a Database
Lesson 7: Multimodal Image Generation
Lesson 8: Agents and Orchestration

Before you start (self-setup)

If you’re following along on your own, complete lesson 0 and lesson 1 first.

Lesson 2 reuses the same Azure OpenAI endpoint, API key, and model deployment from lesson 1.

Why follow-up questions failed

LLMs don’t “remember” by default. Every call is stateless unless you pass prior messages.

So in lesson 2 we move from:

await chatClient.GetResponseAsync(userInput);

to:

await chatClient.GetResponseAsync(history);

where history includes both user and assistant messages.

Build chat memory

We start with:

var history = new List<ChatMessage>();

Then for each turn:

add user message to history
get model response using full history
add assistant response back to history

That loop gives continuity so “What is the worst?” can refer to the previous question.

Message roles matter

By this point we have two roles:

User
Assistant

Then we add a third:

System

System messages are the highest-priority behavior instructions. This is where tone, format rules, and constraints live.

System prompt: from generic bot to Star Wars copilot

The workshop evolves the prompt to:

keep responses concise (optional)
answer in Yoda style
warn about the dark side
respond to “hello there” with only “General Kenobi!”

It sounds playful (because it is), but this is a great pattern for real apps: keep core behavior in one explicit, testable prompt.

Tradeoff: memory vs tokens

Sending history improves quality, but increases token usage and cost.

That tradeoff is unavoidable in chat apps. The practical takeaway: keep enough context for quality, but not so much that costs or latency spike unnecessarily.

A stylized sci-fi scene showing layered chat bubbles orbiting around a glowing holographic AI mentor in a spaceship command room, one bubble labeled by icon only for system rules, one for user, one for assistant. Warm cinematic lighting, high detail, no text, no logos.

Follow along

Workshop source for this lesson: Lesson 2 README.

Next up: swapping model providers, including local models with Foundry Local.

Note: Original workshop repository: jimbobbennett/StarWarsCopilot.

Build a Star Wars Copilot in C# - Lesson 1: Chat with an LLM

Tue, 03 Feb 2026 00:00:00 +0000

I recently put together a workshop called StarWarsCopilot, and I wanted to turn it into a proper blog series so you can follow along at your own pace.

This is lesson 1 of 8.

In this first step we build the foundation: a .NET console app that can send prompts to an LLM and print responses back to the terminal.

Lessons in this series

Lesson
Lesson 0: Self-Setup
Lesson 1: Chat with an LLM
Lesson 2: Chat History and System Prompts
Lesson 3: Model Choice and Local Models
Lesson 4: Tool Calling
Lesson 5: MCP (Model Context Protocol)
Lesson 6: RAG from a Database
Lesson 7: Multimodal Image Generation
Lesson 8: Agents and Orchestration

Before you start (self-setup)

If you’re following the series on your own, start with lesson 0.

For this lesson specifically, you need:

an Azure subscription with permission to create AI resources
Azure OpenAI access in a supported region
a deployed chat model (for example gpt-5-mini)
.NET 10 SDK installed locally

Setup notes for this lesson

In the Azure portal, create an Azure OpenAI resource.
Open the resource and launch Azure AI Foundry for that resource.
Deploy a chat model (for example gpt-5-mini).
Copy:
- your Azure OpenAI endpoint (for example https://<your-resource>.openai.azure.com)
- an API key
- your deployment name (used as model name in this lesson)

Then configure your app secrets:

dotnet user-secrets init
dotnet user-secrets set "OpenAI:Endpoint" "https://<your-resource>.openai.azure.com"
dotnet user-secrets set "OpenAI:APIKey" "<your-api-key>"
dotnet user-secrets set "OpenAI:ModelName" "<your-chat-deployment-name>"

Quick verification:

dotnet user-secrets list

You should see the three OpenAI:* keys before moving on.

What we’re building

The app is intentionally simple:

Read user input from the console
Send it to a model via IChatClient
Print the assistant response
Repeat until the user exits

Under the hood this uses Microsoft.Extensions.AI, which gives us a clean abstraction over different model providers.

Project setup

The workshop starts by scaffolding a console app and installing packages for:

LLM access (Microsoft.Extensions.AI, Microsoft.Extensions.AI.OpenAI)
Azure OpenAI connectivity (Azure.AI.OpenAI)
Config and secrets (Microsoft.Extensions.Configuration.UserSecrets)
Logging (Microsoft.Extensions.Logging.Console)

Then we store model details in user secrets:

dotnet user-secrets set "OpenAI:Endpoint" "..."
dotnet user-secrets set "OpenAI:APIKey" "..."
dotnet user-secrets set "OpenAI:ModelName" "gpt-5-mini"

I like this pattern because it keeps credentials out of source and makes swapping models easy later.

Connecting to the model

The key flow is:

Load endpoint/key/model from configuration
Create AzureOpenAIClient
Convert it to IChatClient
Wrap with logging middleware
Call GetResponseAsync

That “convert to IChatClient” part is the important design choice. It means we can switch providers later without rewriting the entire app loop.

First interactive loop

Once wired up, the app is basically:

while (true)
{
 Console.Write("User > ");
 var userInput = Console.ReadLine();
 if (string.IsNullOrWhiteSpace(userInput))
 break;

 var result = await chatClient.GetResponseAsync(userInput);
 Console.WriteLine("Assistant > " + result.Messages.Last()?.Text);
}

This gets us a working copilot quickly, with great trace visibility from logging.

What you should notice

If you ask:

“What is the best Star Wars movie?”
“What is the worst?”

…the second answer won’t understand context from the first question.

That’s expected. LLMs are stateless unless you provide prior messages.

And that’s exactly where lesson 2 goes next.

A cinematic, retro-futuristic illustration of a developer at a terminal in a starship cockpit, chatting with a glowing AI hologram. Neon blues and warm amber instrument lights, dramatic depth, hopeful mood, high detail, wide composition, no text, no logos.

Follow along

Workshop source for this lesson: Lesson 1 README.

Next up: chat history, message roles, and turning this into a true conversation.

Note: Original workshop repository: jimbobbennett/StarWarsCopilot.

Build a Star Wars Copilot in C# - Lesson 0: Self-Setup

Tue, 27 Jan 2026 00:00:00 +0000

This series came from a taught workshop, so some original steps assumed resources were already provisioned by an instructor.

If you’re following along on your own, this lesson gets you set up so lessons 1-8 work end-to-end.

Consider this your opening crawl: over this series, you’ll learn to build your own Star Wars-inspired copilot, one lesson at a time.

The workshop itself is a hands-on build where you create a Star Wars-themed copilot in C#, step by step, using modern Azure AI tooling.

Aim: build a production-style AI app architecture, not just a one-off chatbot demo.

Goals:

start with a basic chat loop using Microsoft.Extensions.AI
add memory, prompts, and model flexibility (cloud + local)
expand with tools, MCP servers, and retrieval from data
add multimodal image generation and agent orchestration
finish with a reusable, composable architecture you can adapt to real workloads

Lessons in this series

Lesson
Lesson 0: Self-Setup
Lesson 1: Chat with an LLM
Lesson 2: Chat History and System Prompts
Lesson 3: Model Choice and Local Models
Lesson 4: Tool Calling
Lesson 5: MCP (Model Context Protocol)
Lesson 6: RAG from a Database
Lesson 7: Multimodal Image Generation
Lesson 8: Agents and Orchestration

What you’ll set up

local dev tooling (git, .NET 10 SDK, optional Node.js for MCP Inspector)
Azure OpenAI chat deployment
optional Azure AI Inference endpoint
Tavily API key (lesson 4)
Azure Storage Tables + sample data (lesson 6)
Azure OpenAI image deployment (lesson 7)

Local tooling

Install and verify:

git --version
dotnet --version

For MCP Inspector in lesson 5, also install Node.js and verify:

node --version
npx --version

Clone the workshop source

git clone https://github.com/jimbobbennett/StarWarsCopilot.git
cd StarWarsCopilot

Azure account and cost guardrails

You’ll need:

an active Azure subscription
permission to create Azure OpenAI and Storage resources
a spending plan (this series uses billable services)

My recommendation:

use a dedicated resource group for this series
set a budget/alert on that resource group
only deploy the models you need for the current lesson

Azure OpenAI baseline (lesson 1)

Create an Azure OpenAI resource in a supported region.
Deploy a chat model (for example gpt-5-mini).
Copy endpoint, API key, and deployment name.

In your copilot project:

dotnet user-secrets init
dotnet user-secrets set "OpenAI:Endpoint" "https://<your-resource>.openai.azure.com"
dotnet user-secrets set "OpenAI:APIKey" "<your-api-key>"
dotnet user-secrets set "OpenAI:ModelName" "<your-chat-deployment-name>"

Optional: Azure AI Inference + Foundry Local (lesson 3)

If you want to test alternative models via Azure AI Inference:

dotnet user-secrets set "AIInference:Endpoint" "https://<your-foundry-project>.services.ai.azure.com/models"
dotnet user-secrets set "AIInference:ModelName" "<your-inference-model-name>"

If you want local/offline execution, install Foundry Local and test:

foundry model download phi-4-mini
foundry model run phi-4-mini

Tavily key (lesson 4)

Create a Tavily account and set:

dotnet user-secrets set "Tavily:ApiKey" "<your-tavily-api-key>"

Azure Storage Tables (lesson 6)

Create a Storage account and set:

dotnet user-secrets set "AzureStorage:ConnectionString" "<your-storage-connection-string>"

Create these tables:

Figurines
Orders
OrderFigurines

Then seed sample rows using the workshop dataloader (6-rag/dataloader) or Storage Explorer.

Image deployment (lesson 7)

Deploy a GPT image model (for example gpt-image-1 or gpt-image-1.5) in Azure OpenAI and set:

dotnet user-secrets set "ImageGeneration:Endpoint" "https://<your-resource>.openai.azure.com/"
dotnet user-secrets set "ImageGeneration:APIKey" "<your-api-key>"
dotnet user-secrets set "ImageGeneration:ModelName" "<your-image-deployment-name>"

Also worth noting: dall-e-3 has been retired in Azure OpenAI, so use the gpt-image-1 model family for new deployments.

Quick readiness check

Before starting lesson 1, run:

dotnet user-secrets list

You should see at least:

OpenAI:Endpoint
OpenAI:APIKey
OpenAI:ModelName

Add and verify the other secrets as you reach each lesson.

Cleanup when you’re done

To avoid surprise costs:

delete model deployments you no longer need
delete the workshop Storage account
delete the workshop resource group
rotate/revoke any API keys created for exercises

You’re ready for lesson 1.

Note: Original workshop repository: jimbobbennett/StarWarsCopilot.

Can AI help with user personas

Tue, 04 Mar 2025 00:00:00 +0000

I recently recorded an episode of the Pieces AI productivity podcast with Jason Arbon, the CEO of Testers.ai and Checkie.ai.

During this podcast we were discussing how well AI can do at replicating hte actions of humans, and I made the claim that AI will never be as good as humans as it lacks basic empathy. When we consider designing and testing a product, my position was that an AI will never be able to provide those human-level decisions that consider the needs of the individual users.

Example - a mobile app for truck drivers

I gave an example of a mobile app I built at a company many years ago designed for truck drivers to inspect vehicles. The first ideas for the app failed to take into consideration the UX requirements of the audience. Truck drivers in New Zealand and the US (our audience) are typically:

Older as this is not a career young people want to get into
Larger than average due to a sedentary job with poor food choices available at truck stops
Less tech savvy

This is the typical user, and doesn’t represent the entire user base, but we should always design for the least able users

The initial design for our app was sleek and modern, with small UI controls and an experience that used up to date mobile UI experiences. Which was great for the young, healthy, designers and developers, but would fall down when it hit our users.

We got to these considerations by discussing the app with some of the more customer focused members of the company, folks who were former truck drivers themselves, or folks who meet and work with drivers every day. These are the best people to work with - the ones who have access to the lived experience of the customer. This took days to get the data, with many meetings and discussions to drill down to a set of statements that we could verify the UI designs against.

Our users are typically older - which means more potential for poor eyesight. Small controls and text are hard to read.

Our users are typically larger than average - which means larger fingers. A tight UI makes it hard to select the right buttons.

Our users are typically less tech savvy - which means they are not as used to mobile patterns as the young, tech friendly development team. The experience needs to be intuitive and guide them through the actions they need to do.

So it was back to the drawing board with these thoughts in mind, designing a UI that had larger controls, larger, shorter text, and a simpler flow guiding uses through it step by step.

Can an AI empathize like this?

Now in my discussion with Jason, I argued that an AI would not make these design choices. It would not have enough empathy about the user to be able to consider aspects like larger fingers, or poorer eyesight. Jason on te other had disagreed - his experience shows that AI is surprisingly good at building user personas. It has access to such a huge corpus of information around many different characteristics of humans, that it is able to reason across these details and come up with recommendations that are accurate and detailed. AI also has the advantage that it can do this at scale, so when you are considering systems that might have multiple types of users, it can create multiple user personas much faster than a human would. You can then feed these personas into AI testing tools to quickly validate the UX of apps for these different users.

What considerations are there for creating user personas?

When you use an AI to create a user persona, the key point that Jason made was it is better to be vague rather than explicit. The more you guide the AI, the more bias you put into the system. You want an unbiased view, allowing the AI to consider areas you didn’t.

For example, if you prompt an AI with:

“I need to build a UI for an app for truck drivers. What design considerations should I have for the user experience based off truck drivers typically having poor eyesight and large fingers?”

You are going to get a response that focuses on poor eyesight and large fingers. Which is great, as you need to know what UX considerations there are for these situations, but misses out on any other situations you might consider.

Instead it is better to do some prompt chaining. Start by asking what design considerations I should have for the user interface first. Then when you have the list prompt again to get more information on each consideration. This way the AI is not biased by your thoughts in what is important, and it can add its own ideas.

Let’s put it to the test

So based on Jason’s theory, I decided to put this to the test. Let’s see if an AI can help with user personas.

Prompt 1 - get the UX considerations

Prompt 1: “I need to build a UI for an app for truck drivers. What design considerations should I have for an optimal user experience?”

This is a very generic statement. Let’s see how GPT-4o Mini handles this.

When designing a UI for an app aimed at truck drivers, there are several key considerations to ensure an optimal user experience. Here are some important design considerations:

Simplicity and Clarity:

Use a clean and straightforward layout. Avoid clutter and ensure that the most important features are easily accessible.

Use clear and concise language. Truck drivers may be on the road and need to quickly understand what to do.

Large Touch Targets:

Ensure buttons and interactive elements are large enough to be easily tapped while driving. This reduces the risk of accidental touches and improves usability.

Voice Commands:

Incorporate voice command functionality to allow drivers to interact with the app without taking their hands off the wheel or eyes off the road.

Offline Functionality:

Truck drivers may travel through areas with poor connectivity. Ensure that critical features of the app can function offline and sync data when a connection is available.

Navigation and Mapping:

Integrate reliable navigation features that provide real-time traffic updates, route optimization, and truck-specific routing (considering height, weight, and restrictions).

Notifications and Alerts:

Use non-intrusive notifications to alert drivers about important updates (e.g., delivery reminders, route changes) without being distracting.

Customization:

Allow users to customize their dashboard or main screen to prioritize the features they use most often.

Accessibility:

Ensure the app is accessible to all users, including those with disabilities. Consider color contrast, text size, and screen reader compatibility.

Safety Features:

Include features that promote safety, such as a “Do Not Disturb” mode while driving, which can limit notifications and interactions.

User Feedback:

Provide a way for users to give feedback easily. This can help you improve the app based on real-world usage and needs.

Consistent Design Language:

Maintain a consistent design language throughout the app to help users navigate intuitively. Use familiar icons and terminology.

Testing in Real-World Scenarios:

Conduct usability testing with actual truck drivers to gather insights on how they interact with the app in real-world conditions.

By focusing on these design considerations, you can create a user-friendly app that meets the needs of truck drivers while ensuring their safety and efficiency on the road.

So far this is pretty good. The focus is more around considerations when driving - so large touch targets because the user may be driving so can’t devote attention to looking at the app. It also highlighted connectivity - “Truck drivers may travel through areas with poor connectivity. Ensure that critical features of the app can function offline and sync data when a connection is available”. This is something that wasn’t considered initially, so bonus points to the AI.

So we have some good things here, and some we didn’t think off. But we can now take this and dig in further with more prompts.

Prompt 2 - get accessibility considerations

The first response focused on driving, so let’s try a prompt that focuses more on accessibility.

“For this app, what accessibility considerations are there? What are the typical physical characteristics of a truck driver that may impact the design of the app.”

When designing an app for truck drivers, it’s essential to consider accessibility to ensure that all users, including those with disabilities, can effectively use the app. Here are some key accessibility considerations:

Accessibility Considerations:

Color Contrast:

Ensure sufficient contrast between text and background colors to aid users with visual impairments. Use tools to check color contrast ratios.

Text Size and Scalability:

Allow users to adjust text size for better readability. Use scalable fonts that maintain clarity when resized.

Screen Reader Compatibility:

Ensure that all elements of the app are compatible with screen readers. Use semantic HTML and ARIA (Accessible Rich Internet Applications) roles to provide context.

Alternative Text for Images:

Provide descriptive alt text for images and icons to assist users who rely on screen readers.

Keyboard Navigation:

Ensure that all interactive elements can be accessed and operated using a keyboard. This is crucial for users with mobility impairments.

Voice Control:

Incorporate voice control features to allow users to navigate the app without needing to use touch controls.

Clear and Simple Language:

Use plain language and avoid jargon to make the app more understandable for users with cognitive disabilities.

Feedback Mechanisms:

Provide clear feedback for user actions (e.g., button presses, form submissions) to help users understand the results of their interactions.

Customizable UI:

Allow users to customize the interface to suit their needs, such as changing color schemes or layouts.

Typical Physical Characteristics of Truck Drivers:

Age Range:

Truck drivers can vary widely in age, which may affect their comfort with technology. Older drivers may prefer larger text and simpler interfaces.

Physical Stamina:

Truck drivers often spend long hours on the road, which can lead to fatigue. The app should minimize cognitive load and provide quick access to essential features.

Hand Dexterity:

Some drivers may have reduced dexterity due to age or physical conditions. Ensure that touch targets are large enough and spaced adequately to prevent accidental touches.

Visual Impairments:

Some drivers may experience vision issues, such as reduced acuity or color blindness. The app should accommodate these conditions through adjustable settings.

Hearing Impairments:

Consider drivers who may have hearing impairments by providing visual alerts and notifications in addition to audio cues.

Cognitive Load:

Truck drivers need to manage multiple tasks while driving. The app should be designed to minimize distractions and present information clearly and concisely.

By addressing these accessibility considerations and understanding the physical characteristics of truck drivers, you can create an app that is inclusive and user-friendly for a diverse audience.

Now we have more good stuff here. The AI has considered age and dexterity, focusing on large text, simpler interfaces, large targets, and adjustable text. The small worry is the AI is recommending thoughts for folks who use screen readers - unlikely to be the typical truck driver, if your eyesight is bad enough to use a screen reader it is unlikely you would be driving.

It also had recommendations around cognitive load - reducing the impact on drivers who are focusing on the road. Distractions are a major cause of accidents, so anything the app can do to reduce these is a benefit.

Who wins, AI or humans?

So who is the better at empathizing about the human? Are humans better, or as the AI the winner.

As much as it pains me to say it, I think in this case the AI wins. We humans put together a small set of considerations in a matter of days with many back and forth discussions between teams. The AI however was able to replicate what we decided and more in only 2 prompts. About 1 minutes work.

Does this mean we should discount the human and go all in on AI? Absolutely not. AI makes mistakes. In this example the mistakes are few, just references to users who’s visual impairments need accommodations such as screen readers, far outside what is needed legally to drive.

Is this a great contribution? Very much so. To me if I was doing a similar exercise again I would run multiple prompts through the AI, gather data, then work through these with humans to build a final set of design considerations in a very short space of time, saving days.

Please let me know your thoughts? Have you used AI to design a product? Did it have the right design considerations for your user personas? Let me know in the comments.

Build GitHub Actions faster with act

Mon, 23 Sep 2024 00:00:00 +0000

If I said to you that I made 34 commits to a repo, each one with small changes to a single file, with commit messages like “Hoping this works”, “Please work”, and “For f*cks sake, work this time”, you would know exactly why - I’m setting up a CI/CD pipeline!

We’ve all been through this - having to commit our changes to GitHub to test them out as there is no way to do it locally, and ending up with way too many commits because our only feedback loop is to run and check the logs. Compound this with repo permissions where every commit needs to be a PR that is reviewed and approved, and you have way too much work and time spent working on what should be a simple task. If only there was a better way…

There is! This post is all about act, a tool for running GitHub Actions locally, so you can debug and fix them before committing to your repo. It shows act off using the real world example of the GitHub Action I created for the Pieces for Developers C# SDK.

What is act

“Think globally, act locally”

The goal of act is to provide a way to spin up a GitHub Action locally, running them in a container as if they were running in GitHub. It can handle environment variables, secrets, using external actions and more, with a file system setup that mimics what GitHub has. And best of all, this is a free, open source project!

Install act

Installation of act is pretty simple. It uses Docker, so you need that installed (or a compatible container engine), then after that you can install from your package manager of choice. All the options are listed in the act installation guide. I’m a mac user, so used homebrew:

brew install act

Run act

You can run act from the command line. When run, it will run all the jobs in your .github/workflows folder.

act

The first time you run act, it will give you a choice of what type of container you want to run everything in - from huge with everything, to tiny with minimal support for external actions. I chose the middle ground, with a 500MB download (which conveniently enough I was able to do on airplane WiFi).

You can configure which jobs are run by passing events to the command line. This allows you to simulate a pull request, or a push to a branch. For example, to only run actions triggered by a push, run:

act push

Use act to build an action

I recently used act to help me build out a GitHub Action to build and publish the Pieces for Developers C# SDK to nuget. This action needs to do the following:

Be triggered from a new tag
Install .NET 8 (the current LTS version)
Compile the SDK with a release build, passing the version from the tag to the build command
Sign the created nuget package (this is created on build) using a cert and password from the secrets
Push the package to nuget, using an API key that is a secret

Use external actions with act

Act has full support for external actions made available with a uses statement. It will clone the action locally, then run it with docker so that it runs as if it was in a GitHub Action. In my case, this means I can check out my code, and install .NET using the default GitHub actions/setup-dotnet action.

name: Release

on:
 push:
 tags:
 - '*'

jobs:
 publish-to-nuget:
 runs-on: ubuntu-latest
 steps:
 - uses: actions/checkout@v4
 - name: Setup .NET
 uses: actions/setup-dotnet@v4
 with:
 dotnet-version: 8.0.*

Top tip - running an action that installs a large tool like .NET on airplane WiFi is not recommended! Ask me how I know…

Use a tag with act

The next step is to build the nuget package. As part of this, I want to set the version from the tag.

- name: Build
 working-directory: ./src/Client
 run: dotnet build -c Release -p:Version=${GITHUB_REF#refs/tags/v}

GitHub automatically passes this tag through to the action as a default environment variable when run inside GitHub Actions. With act, I can also set environment variables either by passing them into the command line, or adding them to a .env file:

GITHUB_REF=refs/tags/v0.0.7-beta

This allows me to set the tag used for each run, in this example to refs/tags/v0.0.7-beta. The syntax of my build command, ${GITHUB_REF#refs/tags/v} strips the refs/tags/v section, so my build sets the version to 0.0.7-beta.

Use secrets with act

GitHub Actions supports secrets - special values you can configure as the owner of the repo that are surfaced to your actions, but not visible to anyone looking at the repo. These are great for things like API keys or passwords. In my case I need 3 secrets - a certificate to sign my nuget package, a password for the certificate, and an API key for nuget to show it’s me uploading the package.

- name: Get the nuget signing certificate
 id: cert_file
 uses: timheuer/base64-to-file@v1.2
 with:
 fileName: 'certfile.pfx'
 encodedString: ${{ secrets.NUGET_CERTIFICATE }}
- name: Sign the nuget package
 working-directory: ./src/Client
 run: |
 dotnet nuget sign ./bin/Release/Pieces.OS.Client.${GITHUB_REF#refs/tags/v}.nupkg
 --certificate-path /tmp/certfile.pfx
 --certificate-password ${{ secrets.NUGET_CERTIFICATE_PASSWORD }}
 --timestamper http://timestamp.digicert.com
- name: Push to NuGet
 working-directory: ./src/Client
 run: |
 dotnet nuget push ./bin/Release/*.nupkg -k ${{ secrets.NUGET_API_KEY }}
 -s https://nuget.org

With act, you can pass secrets using a .secrets file. This is similar to the .env file, just with your secrets in it. I created a local .secrets file with the secrets I needed.

For the signing certificate, I need a .pfx file, so my way to do this is to encode the contents of the file in base64, upload that as a secret, then use the timheuer/base64-to-file@v1.2 action to convert that secret to a file on the local file system. More details in this blog post from Tim.

NUGET_API_KEY=<key>
NUGET_CERTIFICATE_PASSWORD=<password>
NUGET_CERTIFICATE=<base64 encoded cert>

Always add the .secrets to your .gitignore to avoid accidentally exposing them by adding this file to your repo!

Test it out

With my environment variables and secrets all set up, I was able to test out my action. As always, things failed with each run, but I was able to iterate locally without needing to push my action, raise a PR, get it reviewed and approved, merge it, then tag each time.

The kind of errors I hit were:

Not setting the working directory
Setting the version of the build wrong
Getting the path of the certificate file wrong
Inconsistent case of release and Release for the output folder

All these are easy mistakes to make when you can’t run locally, and would normally take multiple fixes. All fixed locally before I committed my code.

The action ran as if I was inside a GitHub Action - not only did it actually check out my code and build it, it also signed the nuget package and pushed it to the nuget repo. I could see the result with the new package visible inside nuget. This gave me the confidence to raise my PR.

Conclusion

If you are using GitHub Actions, act is an invaluable tool to help you develop your actions and test them locally. Check it out at github.com/nektos/act.

Circles of DevRel part 1 - grow your circle of influence through collaboration

Sat, 20 Jul 2024 00:00:00 +0000

This is the first part of a multipart blog post focusing on looking at DevRel through the idea of circles of influence.

As dev rel people, part of our job is to drive awareness of our product. If you think in terms of sales/marketing, this is feeding the top of the funnel. If you think in terms of the developer journey, this is the discover phase.

However you look at it, it is about helping developers to discover your product, inspiring them to evaluate it, learn more, and ultimately use it in their day to day.

This post focuses on being better at driving awareness by considering circles of influence, and looking at how you can collaborate with others to drive more awareness.

Successfully driving awareness

What does success look like when driving awareness? I’m sure there are many opinions on this, but mine is simple:

How can I make a new audience aware of product/feature/thing?

Emphasis on the new part. To be successful we need to focus on that new. This can be completely new people who are not aware of your product, or existing customers who are not aware of feature X, or new product Y.

Dev Rel circle of influence

As dev rel people, we all have what I think of as a circle of influence, either personally or on our company properties. This might be our social followers, the subscribers to our newsletter, readers of our employers blog, or folks who follow us in other ways. Although we want to grow this circle, in general this stays reasonably static with steady growth as we do more activities.

Each of these content and social locations has a circle of influence, and they all wrap up to your one overarching circle of influence, encompassing personal and company circles.

If we just use these same circles to promote our thing, then we are at risk of preaching to the converted. It’s hard to get net new signups by tweeting to the same circle of people, once we reach saturation there is no more potential customers to reach.

Although it can be good to reach the same circle, for example to keep customers updated on new features, or support community heroes, we risk the echo chamber. The same people attend your livestream and you are spending hours with no quantifiable return on your time as you are not driving awareness. The same people attend your Twitter space, or watch your talk, or read your blog.

To drive more awareness, you need to grow that audience.

Grow your audience

To get more return on your efforts and grow that new audience, you need to do one of 2 things:

Grow your circle of influence
Overlap with someone else’s circle of influence.

The first part is outside the scope of this post, the rest of this post focuses on the second.

Overlap with another circle of influence

The easiest way to two your circle of influence is to leverage the connections you already have to create collaborative content. By that I mean build new content with someone else who has a circle of influence, that way your content reaches their circle as well as yours.

Collaboration on content

As dev rel people, we create content. Some of this content is directly about our product, but very few products are used completely in isolation. Instead we integrate with others. Sometimes directly - using a web framework on a site deployed to a hosting provider, sometimes indirectly - using AI dev tool to write code to talk to a service.

It’s very normal for us to create this kind of content, so the obvious thing to do is to do this in collaboration with a dev rel for that integration. This is a win-win situation. You both have content that is relevant to your product, it’s been easier to create as 2 of you are working on it, and most importantly this overlaps your circles of influence.

For example, at Pieces we have an AI tool that leverages a range of context such as code snippets. If I wanted to write code using a specific cloud service, chances are I have a range of code snippets saved, such as the code to save an object into storage, or post a message to a queue.

If I write a blog post about ‘using Pieces to manage code snippets to interact with cloud service’, not only does it show a real world use case for Pieces, but it also shows a way to interact with the cloud service easier. This is great content for me and can reach my circle of influence when published to my audience. if however, I collaborate on this with a dev rel from the cloud service, the content will then reach their circle of influence as well.

Suddenly they reach my circle of influence, and I reach theirs. This increases the reach of the content, and drives that awareness we want towards new developers. And importantly this is a mutually beneficial relationship. I’m not taking advantage of someone else’s influence, we are sharing our reach.

The Venn diagram of circles of influence

When we think of collaboration, we need to consider 2 things:

Is there a logical content overlap that tells a good story?
Do we have a sufficiently unique circle of influence?

The first point refers to the content. Is it worth us collaborating? If there is no logical story, then no. A blog post on using a flutter UI component with a k8s management pipeline makes no sense. This content won’t resonate with our audience. Instead we want to collaborate with people who are likely to have an interest in using both products. Using Pieces with a popular cloud tool for example, lots of Pieces users want to use various cloud tools, and cloud tool users may want to manage code snippets and can benefit from Pieces. For collaborative content to work there needs to be overlap in the needs, desires, or tech stacks of the audience.

On the other hand, we don’t want our circles of influence to overlap too much. If the Venn diagram of my circle of influence and my collaborators is a circle, then we fail in our goal of reaching new developers.

How do you ensure this? I don’t have a magic formula, it’s all gut feel, but sometimes just looking at the number of mutuals on social can be enough. Harder with things like corporate blog reach, so a lot is finger in the air guess work. The best way is to try it, track metrics, and see if there is net new growth. If you collaborate with multiple people you should be able to measure from new sign ups what topics work well and double down on those.

Fairness of contributions

Now this benefit is not always equal - sometimes the collaborator has a much bigger reach, or a much smaller reach. Personally I believe in sharing the love, so the way you ‘pay back’ working with someone with a much larger audience is by working with someone else with a smaller audience. We are a community of dev rel people, so we help the community without transactionality. If you work for a very small, new, growing company then you don’t have much reach to bring to a FAANG dev rel for example, but pay it back later. Collaborate now, then as you grow, support the next generation of dev rels with small circles of influence.

Conclusion

We all have a circle of influence. To avoid constantly speaking to the same people, consider how collaborations can increase the circle of influence of both you and your collaborators.

So what exactly is Dev Rel?

Thu, 25 Apr 2024 00:00:00 +0000

I work in developer relations - usually abbreviated to dev rel. Despite this being a popular field, with a lot of companies hiring dev rel folks such as developer advocates, there is still a lot of confusion around what developer relations actually is. In this post, I want to give my opinion on what it is, how it can contribute to your companies success, and also a little bit of what it isn’t.

Before we dive in, I do want to note that dev rel can be many, many things. Although there are similarities in what dev rel folks do, I think Kelsey Hightower sums it up best:

Developer relations is about building relations with developers, and covers a multitude of roles. I’ve written this post more from the perspective of developer advocacy, but I do give a call out to the rest of the folks who help build relationships with developers later in this post.

So let’s dig in to this fun topic! This is a long post and a bit of a brain dump, so please share feedback! Oh, and I won’t be covering metrics, that is a book in itself…

How do I describe my job

One lesson I’ve learned from famed tech storyteller James Whittaker is to have a story that describes what you do. This is currently mine:

I help developers be successful with liblab

Prior to my current role, I was at Microsoft, where it was ‘I help developers be successful with microsoft technologies’

This is a very simple statement but a lot to unpack, and I feel it sums up developer relations in very broad strokes.

I help developers be successful with liblab

I help developers - The core of developer relations is helping. I used to work for Chad Fowler at Microsoft, and he gave a guiding principle for the team that I still follow today. “Help first, sell last”. Dev rel focuses on helping developers. By helping we can have an authentic relationship as we are here to help, not to convince you to buy anything. By helping we can solve your problems and make you more productive, or help you ship what you need. Sell last, not because sales are not important - they are, I like getting paid as Lego is expensive, but sales will happen because the product you are advocating for solves their needs and they or their company decide to buy or expand their usage. I’ll come back to sales later.
be successful - Developers often don’t often care about your product or company. Some do (and these are your source of community heros), but most want to build the thing and go home. Anything you can do to help this happen is a good thing. Sometimes this is awareness of your product and how it can help their particular case, other times it’s content to show them how to do the thing, be it docs, videos, tutorials or other types of content. Sometimes it’s hand-holding where you work one-on-one with the developer. As a dev rel, your content helps them be successful so they can finish their work and go home.
with liblab - Insert your company or product here. At the end of the day you work for someone, they are paying the bills, and you need developers to use their technology so you can get paid. If they are successful with your technology, your company gets paid and you get paid. Sometimes the successful with part is just using your technology, but most of the time it is integrating your technology with another technology. For example, liblab is an SDK generation tool, but is most powerful integrated into a CI/CD pipeline so developers can update their API spec, have their SDKs generated automatically, and, yes, go home.

What does ‘be successful’ look like?

As I mentioned earlier - developers want to do their job and go home, so being successful is about empowering developers to do just that by using your tool or product. To achieve this success, they need to take a number of steps:

Discover
Evaluate
Learn
Build
Scale

This list is taken from the Developer Journey from DevRel.Agency which is an excellent resource for planning your dev rel strategy. I like to split this list into 2 sections - outbound and inbound. Advocating to, and advocating for. Discover and to some degree Evaluate are about outbound advocacy, you enable these by advocating to the developer. For sales and marketing folks, this is the top of the funnel. Evaluate, Learn, Build and Scale are more focused on inbound advocacy, or advocating for the developer. This is the middle to the bottom of the funnel.

Outbound advocacy/Advocating to - providing a mix of first and third party content to drive awareness. This is how you help your company be successful.
Inbound advocacy/Advocating for - providing first party content to help the developer learn, and providing product feedback to help the product be successful. This is how you help your users be successful.

In a perfect world you go where developers are (Discover), and bring them back to you (Evaluate, Learn, Build, Scale).

At every step you will lose developers, just like a typical sales funnel. Your goal as dev rel is to ensure that as much is in place as possible to avoid developers falling out of the funnel.

Discover

To be successful with your product, developers need to be aware of it! Put simply if they don’t know about your product, they will never use it. This is your job as dev rel to raise awareness. You do this by going where developers are, and raising awareness of your product.

Now who are these ‘developers’? A lot of that depends on your product. Ideally you should define one or more ‘ideal customer profiles’ or ICPs that provide a rough summary of who your users might be. You can then target content to these.

For smaller companies, this awareness is around both product and company. For larger companies, this awareness is usually around product. For example, with liblab I have to raise awareness of the company and the product. At Microsoft I was raising awareness of individual products or product areas - after all, most developers have heard of Microsoft.

Going where developers are, and raising awareness of your product - again, a ‘simple` sentence that has a lot of meaning.

Go where developers are

Developers will not come to your docs, blog, LinkedIn page or Discord server if they don’t know about you. You need to share your message in places where those developers are. There are a lot of places, some where developers are seeking answers, some where discovery is more organic. The main thing here though is to be authentic! Help first, sell last.

Conferences - Developers come to these to learn, often with specific problems in mind from their job. The reach is small, but you can have deeper conversations, learn what their problems or use cases are, and show how your product can help. Conferences are very expensive when you count travel, time and sponsorship, but can lead to some big deals if, and only if, you meet the right people there. The return on investment (ROI) can vary dramatically. It’s also hard to measure the ROI as someone who you didn’t track as a lead may be responsible for a huge sale!

Conferences can also help with other parts of your job, such as validating your customer base for product market fit, and testing out product messaging
Workshops - Workshops are an interesting way to reach developers. Often these take up a large amount of a developers time, or even cost money. If a developer is wiling to make that kind of commitment, then they are interested in solving a problem that your product might help with. Small turn out, but very engaged.
Meetups and smaller community events - These are events usually with a technology focus. If your product fits into that technology (such as talking about Python SDK generation at a Python meetup), they can be a relatively cheap and low lift way to put your message to the audience. In person meetups might come with travel cost and maybe pizza and drinks, remote ones can be done from home. The audiences are typically small, and the ROI can vary. Meetup organizers are always looking for speakers - they are the commodity in the shortest supply, so it is relatively easy to get a speaking slot.
Podcasts and live streams - There is a wealth of podcasts and livestreams out there, some focusing on specific technologies, some very general. The hosts of these shows are always on the lookout for new guests and in return let you promote your product.
Blog posts - Although your first party blog might be ok for organic search, third party blogs are a great place to put content. They likely have a large readership, and allow you to put your message across to their audience. I can cross post to a popular tech blog from the liblab blog and get 100x or even 1000x views in a few days!
Tech communities - If you are a large company, you may already have a first party tech community that you can promote your product to. For smaller companies there are third party communities you can reach out to. Get involved, show how your product can help, and that can drive awareness.
Organic search - There are a huge number of dark matter developers who will not see your product at conferences or communities, instead they will search for a solution to their problem. This is where good web properties with good SEO can help.

Raise awareness

To raise awareness of your product you have to help first, sell last. What I mean by this is when you go where developers are, they have problems to solve so they can go home. You have to ensure the message you have is about how your product can solve their problem. This is the focus of your content, and where storytelling is important. Tell the stories of how your product can solve similar problems and you will hook the developer in.

Bring an authentic message - Your product is the hero in a story starring the developer who is trying to defeat the monster that is a problem. The developer wants to fight the ogre in the swamp that is manually creating and maintaining SDKs, and on their journey to the ogre’s lair discovers liblab which gives them the magic to do this. OK - not a talk you would give, but linking it back to the classic storytelling - a protagonist, an antagonist, a journey, a MacGuffin, and a happy ending. Your product is the MacGuffin, your content is the guide on the journey.
Know your audience - The most important thing about content creation is knowing your audience. If you are reaching out to a Python community, then don’t show off your TypeScript SDK. Talking about API governance at a conference focusing on UI design is a waste of time. Pick the places where developers are who your product can help.
Reuse content - Content creation is hard and time consuming. Always think about how you can re-use content to get more mileage. You have a tutorial on using yor product with AI? Can you make a workshop, video or conference talk out of this? You are going to a conference to give a talk? Are there any meetups local to the conference where you can give the same talk at?

Evaluate

Once developers are aware of your product, they want to evaluate it. This will be where you can start to bring them back to your first party content.

To evaluate your product, a developer may do one or more of the following:

Consume more content to see if your product matches their use case
Sign up for/install your product
Work through a ‘hello world’, a tutorial, quickstart or other getting started guide
Join your community, such as your Discord server

Most of this is content. If you are a developer advocate, you might be responsible for creating this content, or you may have a technical writing team to do it for you.

The goal here is to get the developer quickly to a point where they feel your product can solve their problem. You need to provide a mixture of content so that the developer can see something that is close to their use case so they can do a relevant evaluation. Obviously this is hard as there are infinite use cases, but you should be able to pick a few broad topics that cover most, especially if you have defined a number of ICPs.

To evaluate, you need as few gates as possible. Yeah, an email address to sign up is normal, but if a developer has to talk to a person to get a trial then it won’t happen. Your product needs to be self service, with a free tier that is enough to fully evaluate. As a dev rel, you are responsible for product feedback, and that includes feedback on the size of this trial. As an engineer yourself, can you evaluate with the available trial? If not, it needs to be changed.

Learn

Once the developer has evaluated your product an decided it is for them, they need to learn more. This is where good documentation is vital. If the developer cannot find out how to solve their problems, then they will go elsewhere. As a dev rel you may be responsible for the docs, or you may have a team to do this. At the very least, you are responsible for product feedback, and that includes feedback on documentation. Yes - documentation is a product and anyone who says different is very, very wrong.

If it’s not documented it doesn’t exist.

In dev rel, you should always be reading and using your docs. Any thing missing or wrong - you should work to get it fixed, whether that is writing those docs yourself, or working with the documentation team to get them fixed.

Your documentation needs to help the developer learn all they need. This means you need a mixture of docs - reference documentation, tutorials, concepts, all packaged up in a way that quickly allows the developer to get their answers.

Build

After learning what they need, the developer will start building with your product. Learn and Build usually go hand in hand - reading docs whilst building out with your product. Building is the phase where they are in proof-of-concept (PoC), or even production mode. By now they may have paid you a little, or have approval to pay you once their trial runs out. For your company to get the bigger payout, they need to be successful.

This is where support, community and documentation are important. You don’t want the developer to get stuck on the build out, not complete, and not convert from a free trial. You may have a support team to help, but dev rel folks should be monitoring the popular support requests and use that as feedback on what content to create. How you do this depends on your support team - maybe a meeting once a month to get the most popular issues, then you write docs for these.

Developers like communities - they want to ask questions on stack overflow or a Discord server. Again, in dev rel part of this sits on your shoulders. Even if you have a support team to do this, you need to be aware of the trends to build content. If you don’t have a support team then set aside some time each day or week to scan stack overflow and your community tools to provide help, and encourage engineers to do this to!

If you see developers being active with your product then dropping off, don’t be afraid to reach out to them. Help first and all that. ‘Hey, I saw you were using our tool then stopped at this point, are you stuck on something that I can help with?’ type messages can achieve a lot. Often providing a simple answer or guidance can unblock them and get them back building. And of course, update this into your documentation!

Scale

By scale, I mean developers scaling out what they have built. Using your product more, using it in different ways, using different parts of it. Taking what they have built and moving it to production.

Hopefully by this point, your job is pretty much done. The main responsibility you have here is to ensure the content is available to show how to scale. For example, with liblab, scale to me is taking their SDK generation and setting it up in a CI/CD pipeline so that they can forget about it and get new SDKs when they update their API specs. This is a relatively complicated thing to set up (relative to generating an SDK as there are a few moving parts, but not complicated compared to kubernetes).

For developers to scale like this, they need to know how. Not just the steps to take, but the why of each step so they can apply it to their use case. For example, if you have developers deploying your product to public clouds, then you want docs on how to do it on AWS, Azure and GCP. If you only have AWS docs and they use Azure, then they will struggle to scale.

What’s after scale?

At this point the developer journey is not done. Yes they have their problem solved and they can go home, but a new day is a new problem. The cycle starts again with whatever new problem they have, or whatever new product you have for them.

Scaling dev rel

So far I’ve talked about the journey developers take and how dev rel fits in. The biggest issue dev rels have with this is one of scale. You likely have a small dev rel team, but a huge user base, so how do you scale?

As a note, and I did say I won’t cover metrics here, you can only scale if you can show you are scaling. And that means metrics and tracking. You do need to be able to show that scaling is working.

There are a number of ways to scale, and here are a few.

Reduce, reuse, recycle

A lot of dev rel time is spent on content, be that writing talks, creating videos, writing sample code, or writing blog posts. The most effective thing you can do is re-use as much as possible. Spend less time creating, more time sharing. The downside is that a lot of dev rel folks love building the new shiny thing, but you need to avoid the temptation to create a new talk every time.

For example - back at Microsoft I created a sample mobile app with Xamarin to show some of the Azure AI services. Here’s how I scaled it:

A GitHub repo with code samples shared on social and blogs
Feedback on the docs based off building this app
A video showing the app and code on a popular show
A workshop that taught people to build the app.
- Gave this to team mates to run
- Ran this at multiple events

One code sample became a lot of content, reaching probably at least 500 workshop attendees - and if someone is attending a workshop then they are very engaged and less likely to fall out of the funnel. Oh, and in the case of this workshop, one group of attendees went on to build an app in production for the Oscars!

Online vs in-person

This is a thorny topic as there are very different opinions and data, but online reaches orders of magnitude more people than in-person for the same amount of effort. I can go to a conference, spend 2 days traveling and 2 days at the event to reach maybe 100 people with a talk. Or I can give that same talk 10 times in less time on-line to different communities and reach thousands. Microsoft saw this with MS Build - instead of 6,000 in-person attendees then had 160,000 online.

Now there are lies, damned lies, and statistics - so how important is this difference?

Well the jury is still out on that. In person means more engagement. If someone comes to a talk they are genuinely interested. If they are watching online are they actually watching? Or is it on in the background whilst they do something else.

I wish I had hard data for this, but I don’t so my gut feel is a mix of both. Having seen a good ROI from in-person events that are targeted to a relevant audience (for example talking about SDK generation for APIs at an API conference), there is benefit to in-person, but it doesn’t scale like on-line.

Reach more developers by going where developers are

To scale well you need reach. I’ve seen for example the same post get 3,000 views on DZone, and less than 25 on dev.to. By putting your content where developers are it helps scale. This also means your content needs to be of the right type for where those developers are. There’s always debate on blogs vs video, but my take is both and measure. YouTube is the worlds number 2 search engine, so a well constructed video with a good description and title can reach a massive audience. But then there are those who prefer to read, or skim read, or want a TikTok.

And when I say measure - try to get good data. Views on a blog may mean a lot of engaged developers, or lots of quick view and bounce. Are you seeing folks following links from the blogs to your properties (you are tracking this right?).

Same with YouTube views. I remember back at Microsoft a load of congratulatory emails and lots of talk about an amazing video that a colleague did on generative AI that got 40,000 views. Even went way up the chain. Ignoring how 40K is a small number for a multi-trillion dollar company, no-one bothered to validate the views. If they had, they would have noticed that most of the 40k views were around 5 seconds and were from ads. Yup - there was an experiment done to run videos as ads to boost views, and that is why this had so many. Obviously for political reasons, when this was pointed out it was hushed up…

So, go where developers are, validate your reach there, and use that data to help scale.

Follow trends

Another way to scale is to create content to ‘go viral’, that is follow trends, If a topic is getting a lot of interest, see how you can apply your product to that topic. LLMs and Generative AI is big, so I created a video using a popular AI framework from Microsoft just as that framework was getting promotion by them. It has my highest personal video views! This was simply by riding the promotional wave from Microsoft.

Community

Community is how you can really scale! A good community becomes almost like a volunteer dev rel team. Having been a Xamarin MVP and a Microsoft MVP I know well how community will promote a product in their own time and at their own expense - I’ve spent my own money on travel just to talk about a technology owned by a trillion dollar company.

Community helps with the scale simply by shear volume. A community of 10 heros can reach substantially more that 1 dev rel, including reaching geographies and time zones that are not easily available. A community hero in China giving a talk on your product in Mandarin can reach developers an English-only speaking developer advocate in the US simply cannot reach. The more you grow your community, the more they will promote your product.

The advantage of your community is they are already engaged. people join your community because at the very least they are aware of your product, though more likely a user. Folks don’t discover your slack by chance and hang out.

So how to do you grow your community? Well a ‘community’ has 3 types of members:

Passive - these are the folks who join your slack to ask a question, maybe follow you on social and leave. You need to support them as they are users, but they won’t help you scale. What they are helpful for is product feedback - the questions they are asking, should they be answered in the docs? Are they reporting bugs or fewture requests?. Your goal here is to make them active, but be aware most of them simply won’t be.
Active - these are the folks who are active in your community space. They might be answering questions on your Discord, or maybe writing the odd blog post on your product. They can help you scale support, as well as driving some awareness. Your goal here is to convert these to heros.
Heros - these are your very active, highly engaged, community superstars. These are community members who actively promote your product. Maybe then run a user group, or regularly speak at events about your technology. Maybe they write blog posts, or create videos. Their message is the most authentic - they are talking about your product because they like it, not because they are paid (like you). Your goal here is to support them as much as possible. At the very least they should have free access to your product, along with swag. Ideally you should provide swag for them to distribute like bags of stickers, custom rewards like personal shirts, and a public way to highlight them. Recognition is usually the most valuable currency here, not just recognizing then publicly, but by gathering feedback from them. Getting time with product or engineering is a great way to not only make them feel special, but also can give you important feedback. If you have the budget, provide them sponsorship such as paying conference travel, or buying pizza for their user group. You can also help provide them with content, such as demos and slides for a talk.

Your goal here is to support them as much as possible. You want them to remain heros as long as possible, and not turn them into heros for your competition.

I was listening to an episode of Dev Relish today that described community as a Tamagotchi. For it to live and thrive it takes regular feeding. Not much effort, but small and continual. You need to build that muscle of regularly ‘feeding’ your community.

What else is dev rel?

I feel the focus of this post has been more on the advocacy side of dev rel, as that is the area I am in. I do want to acknowledge that there are many other facets to developer relations that help build a relationship with developers:

Technical writing teams - dev rels sometimes own and write docs, other times there are dedicated teams. And this is not just docs, it can be online learning tools or other such learning materials
Content creators and supporters - some companies have dedicated studio teams, or storytellers, or demo writing teams
Developer-focused engineers - some engineers focus on helping developers directly, rather then by building products. This includes generating SDKs, or building helper scripts, sample code, or other tools
Community managers - in an ideal world you would have a community manager to feed the Tamagotchi. These are folks whose skills lie in managing communities, such as program management.
Marketing - marketing spread the word, and can be your friend. They also usually have money which helps with swag!
Events teams - if you are running an in-house event, or attending an external one, an event team can make this a much smoother process

I’m sure there are more. I know some will say ’this is not dev rel’, but it is. It may not be dev advocacy, but dev rel is a whole world more than that.

What is not dev rel?

So far I’ve talked a lot about what is dev rel and how you can scale. Lets take a second to talk about what is not dev rel. This is another topic where there are many opinions, but my (potentially 🌶️) opinion is:

If you are personally responsible for getting money then it is not dev rel

Now we all want to get paid, and really the end outcome of dev rel is more customers, so dev rel is part of the sales funnel. But to me the line is when money is involved, such as negotiating contracts, commissions etc. then that is sales. Help first, sell last - to authentically help someone it needs to come from a place of wanting to help, not from driving more sales. An authentic dev rel should be willing to help you spend less by optimizing what you use, or telling you outright if the product will not help you. Having worked in places where sales will sell something that doesn’t exist (yup, happened to me), I’ve seen that money and commission are not a good driver of the right behaviors, and do not help with authentic conversations.

I’m not covering metrics here, but dev rel should never be measured sales. Yes - tracking dev rel qualified leads is good, but not ‘dev advocate X made this $100K sale’

Help first, sell last - and let someone else sell.

Conclusion

I hope this brain dump sheds some light on what I think dev rel is. As with anything, ask 10 dev rels what it is and you’ll get 12 answers at least. There’s so much I’ve missed - I literally could write a book on this topic, but I won’t - I’ll save that for the experts.

Remember: Help first, sell last. Authentically go where developers are and bring them back to your product, content and communities.

I’d love your thoughts in the comments, or reach out to me on social: linktr.ee/jimbobbennett.

How to correctly review a pull request

Mon, 01 Apr 2024 00:00:00 +0000

One of my popular sayings is that development is a team sport. We don’t code in isolation, instead engineers are part of a team that delivers value to users and customers, often made up of software. And for a team to deliver its best code, there needs to be shared ownership of the code. This means that more than one developer is involved in each code change to ensure that it is correct, appropriate, and that the knowledge of this change is shared amongst the team. While there are many techniques for this (such as pair and mob programming), the most popular one is a simple pull request, or PR.

What is a pull request?

A pull request, or PR, is a git feature that allows you to request that the owner of some code pulls your changes. You make a code change on a fork or branch, and raise a pull request to merge your code into the golden source, origin, or upstream repo. Essentially this is how you get your code published to the team ready to be deployed to production.

The aim of a pull request is to get another set of eyes over your code. Before a PR is merged, it is reviewed. The reviewer can check the code for correctness, style, if it actually implements the feature it’s meant to, things like that. This is the opportunity for a second pair of eyes to catch any errors, for knowledge sharing so more than one member of the team understands the new code, and a chance for more senior developers to help guide and upskill more junior developers.

The problem with PRs though, is often the reviews are done badly. The infamous LGTM 👍 (looks good to me). Especially with large PRs that take a lot of time.

So why are we so bad at reviewing PRs?

The reason is simple - time. Reviewing a 5 line change is potentially quick, so a reviewer can check it, and ask for changes in a short space of time. Give a developer 100 files then the review is often little more than a cursory glace as there isn’t enough time to do a full review. After all reviewing code isn’t delivering features!

Having seen some very bad PR reviews in my time, and poor code released because of it, I thought I’d scribble down some of my top tips for doing PR reviews right. Add any more thoughts to the comments!

How to do a better job of reviewing PRs

1. Allocate time to do it

This is often something that you as a reviewer can’t do, but something you should advocate for and your leadership needs to understand. PR reviews take time. If you expect an engineer to be 100% on feature work, not only do you risk burnout, but it means there is literally no time to review PRs. Everything, no matter how bad will be LGTM 👍, merged and cause problems downstream.

This makes sense for engineers - there’s not time to review a PR to catch a bug, but it’s easy to get time allocated in a sprint to fix the bugs that get through. Your processes drive poor engineering practice.

Ideally an engineer should be 60% feature work, 20% run the business, 20% spare (vacations, sick time, dealing with weird issues that reduce your ability to work like your OS shitting itself). The 20% run the business should be enough to review PRs, and if not - allocate more time (or move to pair programming, but that takes more to convince management who see people as resources with linear delivery capability).

Now you have time to do a proper PR review, next you need to review the ticket!

2. Review the ticket

All good PRs start with a ticket of some description. This is (typically) a ticket in Jira that details the work that is done. But these are not code, so why do you care if you are reviewing a PR? Well the ticket explains the intention of the change. What bug is being fixed, what feature is being added. Without a good detailed ticket, how can you very the code works?

So what makes a good ticket? This is blog post in its own right, but here are some thoughts:

It exists. Simply having a ticket is a start. No ticket, then no change. All PRs without a ticket should be rejected as how can you ensure the change is correct?
It explains the why of the change. When you know why you can think about the implementation from the perspective of the end user. Helps when you think about the test cases or ways to test the code.
It explains the change in as much detail as possible. Move a button? Ok, where, by how much. Add a new API? What endpoint, what is the request and response body?
It defines some test cases. In a perfect world these should be exhaustive, but it should at least direct some ways to verify the change. Add a feature to calculate X - he are some sample inputs and outputs.
It defines any impact on other parts of the system. Does it require a downstream change? Should documentation be updated?

Once you are happy with the ticket you can review the code. Not happy - punt it back to the developer or product owner to get it to a good state. Sometimes just by doing this you will catch things the developer missed and get bugs fixed or missing features implemented before you have even looked at the PR.

3. Does the code work?

Code should work. The ticket defines the change, and the code should implement it. Before reviewing the code, I like to take it for a spin and see if it actually does what it is meant to do in the context of the whole system. Yes, this usually means some manual testing but again you can catch issues quickly. Not always easy, but there are usually ways and test harnesses to test features in components of a larger system. This is where a good ticket also helps - it can define what you need to test, expected outputs for inputs, and give you a good direction to test edge cases.

But what about unit tests? I’ll cover these next, but the presence of unit tests doesn’t always mean tha the code works. I’ve had situations when reviewing a PR where all the tests passed but the code didn’t do what was needed when I tested the whole system. Multiple times I showed the engineer the issue, they added unit tests, told me it worked because their unit tests passed, but it didn’t work.

For example, if you have an add function and write the following tests: add 2 and 2 to get 4, add 3 and 1 to get 4, and 0 and 4 to get 4. Do these tests show the code works, or that you have a function that always returns 4?

There’s no point in reviewing code if it doesn’t work.

4. Does the code have automated tests?

The next step with testing - what about the automated tests? Does the code have unit test, integration tests, UI tests etc. If not - should it (probably). If yes, do they run, pass, and actually test the code?

I find reviewing the test code first is a good thing to do. If there are obvious test cases missed, edge cases not considered, or poor tests, then you want these fixed before you review the core code.

Can you see in the PR that the tests have run and passed? As above - no point in reviewing code that doesn’t work. A PR should run as many unit test as is practical, and there should be guards that block merges if any tests fail.

5. Can you understand the code?

Now it is time to review the code. The first thing to review is do you understand it? If you don’t understand the code and what it does then you can’t review it. It should be reasonably obvious to someone who understands the system and context of the changes. It should also be readable. Does the code have sensible names for variables, functions, and classes? Are there code comments explaining the why of the code?

Anyone who ways code comments are not needed is wrong. Yes - well named things explain what the code does, but comments explain the why. There is no need to comment a getter to say it gets a value, but you want to explain why you are getting that value in the code and doing things with it

Readability is important in code. I know some developers like to play code golf and solve a problem in as few lines as possible, but sacrificing readability is a terrible thing. You are writing code so humans can understand it, so don’t be clever, be clear, concise, and obvious. When reviewing code is it clear to the reader what it is doing and why? Your code should be obvious to the most junior developer on the team - remember code is owned by the team, not the engineer who wrote it, so needs to be accessible to everyone.

6. Is the code good?

Lastly is the code ‘good’ - and by good I mean following team standards. This could include coding standards (tools help here), but also does it follow conventions, is it implemented in a consistent way with the rest of the code base.

For example - if your code writes a message to the user, is this message defined in a constants file with other messages for i18n? Does it conform to a11y standards? Is the naming convention consistent?

This is often subjective, so having a good team agreement with coding standards can help with this.

if it is all good - approve that PR! Any issues, then it’s time to leave review comments.

Commenting on PRs and requesting changes

You’ve reviewed the code and it’s time to leave some comments. Although this post is about reviewing, here are some thoughts on leaving review comments.

Be kind. Don’t be an arse, don’t be Linus Torvalds. The person receiving these comments is human and a team mate. Be kind and constructive with all your comments.
Provide detail in the comments. If something needs to be changed, explain why (e.g. move this to a constants file as per our standards).
Understand that everyone codes differently and personal opinions have no place. You would have written it differently? Well if the way it is done is ok, then don’t impose personal opinions on the implementation. If you ask 20 developers how to do something you’ll get 30 responses, not all are wrong.
For more junior developers, this is an opportunity for learning and growth. Give guidance with changes that help with that growth
Use things like suggestions for small changes like typos to make it quicker to apply changes

Go review PRs

This was a quick and dirty post to get some thoughts down on this. Let me know your thoughts in the comments!

How to generate pre-signed URLs to get and put objects in S3 with Boto3 in Python

Thu, 01 Feb 2024 00:00:00 +0000

TL;DR: If your put_object boto3 pre-signed URLs are failing with access denied errors, add config=Config(signature_version="s3v4") when creating the boto3 client to get a working URL.

I was recently working on some Python code to interact with S3 buckets, where I needed to be able to generate some pre-signed URLs for S3 objects. These are URLs that contain a built-in time-limited credential to allow access, meaning that you can give this URL to anyone and they can read an object, or write it without access to your AWS account or S3 bucket - they only have access to that one object.

In Python there is a helpful library called boto3 from Amazon that can be used to interact with AWS services. This includes S3, and it has a method to generate pre-signed URLs for objects.

Get started with boto3

Set your credentials

You can only access S3 if you have the correct credentials. These are your AWS access key and secret key, and need to be set in your AWS credentials file using the AWS CLI:

aws configure

Or you can edit the ~/.aws/credentials file directly:

[default]
aws_access_key_id = YOUR_ACCESS_KEY
aws_secret_access_key = YOUR_SECRET

Install boto3

To use boto3, you need to install it in your current Python environment. This can be done with pip:

pip install boto3

Obviously using a virtual environment or a development container is a good idea, but I’ll leave that up to you 😁.

Once done, you just need to import boto3 into your Python code:

import boto3

Then you need to create a client for the service you want to use. In this case, S3:

region = 'us-east-1'
s3 = boto3.client('s3',
 region_name=region)

Clients are regional, so you need to specify the region you want to use. In this case, I’m using us-east-1.

Create pre-signed URLs

Once the client is created, you can use it to generate pre-signed URLs for objects in your S3 bucket using the generate_presigned_url method.

Create a get object pre-signed URL

The first case I needed was for someone to be able to download an object from my S3 bucket. In AWS, this is the get_object client method.

Here is the code to generate a pre-signed URL for getting an object in an S3 bucket:

url = s3.generate_presigned_url(
 ClientMethod='get_object',
 Params={
 'Bucket': 'my-bucket',
 'Key': 'my-object'
 },
 ExpiresIn=3600
)

This takes the client method - get_object, the parameters - the bucket and the object key, and the time the URL is valid for in seconds.

This gives you a working URL that anyone can use for the object my-object in the bucket my-bucket that is valid for 3600 seconds, or 1 hour. You share that URL with someone, and they can download that one object only.

Create a put object pre-signed URL

The second case I needed was for someone to be able to upload an object to my S3 bucket. In AWS, this is the put_object client method.

Now, in theory this is the same as the get_object call just with a different client method:

url = s3.generate_presigned_url(
 ClientMethod='put_object',
 Params={
 'Bucket': 'my-bucket',
 'Key': 'new-object'
 },
 ExpiresIn=3600
)

As we all know, theory and practice are the same in theory, but not in practice. This code will generate a URL, but when you try to use it, you will get an XML document with an access denied error.

The request signature we calculated does not match the signature

This took hours to get to the bottom of, but after plenty of S3 reconfiguration and some googling, I found the solution in an answer buried in this stack overflow question

It seems this configuration option is not well documented. It is mentioned in the boto config docs but without any details on why you need a different signature version for put_object URLs, or what the default is.

So, to generate a working pre-signed URL for put_object in S3, you need to add the signature_version to the Config object when creating the client:

from botocore.config import Config

s3 = boto3.client('s3',
 region_name=region,
 config=Config(signature_version="s3v4"))

Once you have done this, the code above to generate a pre-signed URL for put_object will create a URL that works as expected. You can now share this with anyone and they will be able to upload a new object to your S3 bucket, with the key given in the params.

In the example above, they would be able to upload an object to the bucket my-bucket with the key new-object. Again, the URL has an expiry, so is valid for 3600 seconds, or 1 hour.

Conclusion

Saurav Sakar, you are my hero! If you are creating re-signed URLs, make sure to set the signature_version to s3v4 in the Config object when creating the client.

Connect to localhost from inside a dev container

Mon, 20 Nov 2023 00:00:00 +0000

I do a lot of work in dev containers (for example, using liblab inside of one), often hosting APIs. One problem I used to often hit is how can I host an API in one container, and access it from another dev container? This post shows you how.

The problem - ’localhost’ in a container

So the issue is this:

Dev container A exposes an API on port 8000, and this is forwarded to localhost
I can access this port at localhost:8000 either inside dev container A, or from my local machine as the dev container forwards the port
Dev container B needs to access the API

The problem here is dev container B has no access to my local machine. Docker forwards the ports from dev container A to localhost automatically as soon as I run the API. But ports from my local machine are not forwarded into dev containers.

If Dev container A accesses localhost:8000, it gets the API running on Dev container A
If my local machine accesses localhost:8000, it gets the API running on Dev container A because the port on Dev container A is forwarded to localhost, as far as the local machine is concerned, 8000 is open and accepting requests.
If Dev container B accesses localhost:8000 it will fail, as there is nothing running on port 8000 inside that container.

So how can one dev container access ports exposed by another?

Enter the host.docker.internal network

Docker has a fix for this! It exposes a ‘special’ DNS called host.docker.internal that essentially gives to access to the local machines network. Rather than access localhost, you access this named device instead.

So from Dev container B, you access host.docker.internal:8000 and boom! Dev container B can access the API from Dev container A!

For example, you are testing out an SDK created against an API you are running on port 8000 via a dev container (for example, the liblab llama store), and you want to take advantage of the dev containers feature of liblab to open our SDK in a container. You can set the baseUrl in your liblab.config.json file to http://host.docker.internal:8000, then open the generated SDK in its dev container!

{
 "sdkName": "llama-game",
 "specFilePath": "http://localhost:8000/openapi.json",
 "baseUrl": "http://host.docker.internal:8000",
 "languages": [
 "python"
 ],
 "createDocs": true,
 "customizations": {
 "devContainer": true
 }
}

Done! The SDK now defaults to use the docker internal network for testing.

class Environment(Enum):
 """The environments available for this SDK"""

 DEFAULT = "http://host.docker.internal:8000"

Don’t forget to set it to your production URL once you are ready to publish.

Avoiding automation to keep your job

Mon, 06 Nov 2023 00:00:00 +0000

Will technology, automation, or AI take our jobs? This is something we hear a lot at the moment - but it’s always been this way. Right back at the start of my career I automated something that was canned the moment I left as folks were afraid of their job.

Some background to why I am writing this post. I was reading a tweet today from Michelle Bakels that was a cautionary tale from her enterprise days.

Cautionary tale from enterprise days re: AI/automation:

Company touted big Innovation Week where IT interviewed other depts (accounting, marketing, etc.) to find workflow optimizations for the business.

After 3 years, the other depts shut down when IT would come for interviews.…
— Michelle Bakels (@MichelleBakels) November 6, 2023

I’m not going to dump the entire text here - go read the tweet as it’s fascinating, but the TL;DR is folks are scared of telling IT what they do in case it is automated and they are out of a job.

Will technology replace <thing>?

This is something that is constantly talked about in the tech world, though feels like it has really come to the surface recently with the rise of AI. Will technology replace us? Will AI take our jobs (one of the catalysts of the recent SAG-AFTRA strikes in the TV and movie industry)? Will self-driving cars replace taxies? Will cars replace horses? Will movable type replace handwritten books? The list goes on.

Rather than dive into this discussion, I thought it would be fun to tell a story of my own - of how I automated something during an internship, and how it got killed the minute I left as folks were worried it would replace their jobs.

Jim gets an internship at Marconi

This was just after I graduated in 1999 - I started working on a summer internship at Marconi (plays the mamba, listen to the radio) in their underwater weapons division. This was probably the start of my route towards being a pacifist and hating all weapons and the military industrial complex.

My job was to analyze torpedo data. Marconi built these $1M Spearfish torpedoes that didn’t work very well, and they would do trials on them in various parts of the world (Faslane in Scotland, deep water testing in the Caribbean). The testing torpedoes would record data about their decisions, as well as telemetry such as depth, speed, location etc. They would also record radar data and what they identified in the way of ships. The goal was to drop a torpedo in the water and send it at a ship, then gather the data afterwards to see if it did the right thing. Not quite testing in production as the torpedoes had no explosives, but the closest you can get.

The reporting process

Once the trails were over, someone had to analyze the results, and this was the team I was on. The process was horribly manual:

Remotely log into a mainframe and request a printout of the data from the torpedo trial as a hex dump on green and white continuous feed paper.
Work through the hex dump using highlighters to ‘read’ what was happening. Hex codes marked the steps in the output, and each one had so many bytes of information. For example, one hex code was for direction and you knew that meant the next X bytes were the direction information, then the next character was the next set of data. Very laborious and error prone
Enter all this data into an Excel spreadsheet showing each step
Extract location and direction data and put this into another page on the spreadsheet
Generate a chart to show the torpedo’s movements
Create a word doc with all the steps that the torpedo did, along with the charts.
Actually do the analysis and look for issues for the software teams to investigate

(There are probably more steps which I have forgotten as it was 24 years ago)

Apart from the last step which actually used your brain, the rest was repetitive and dull. It could easily take a week or to to get a report ready to be able to do the final analysis step. And there was a whole team doing this, most of them ex-Ministry of Defense folks who just wanted to get their pension.

In comes Jim with an idea!

So I had an idea! I was an engineer after all. I could automate all the things, I’d be the hero, save the company money and everyone would love me.

via GIPHY

So I set about doing this whilst I was working on a report.

Automate all the things

Firstly I had to automate getting the data - can’t do this from a printer. Luckily the mainframe that I logged into to print could just dump the data to the screen. How could I get this in a way I could do things with it? Excel!

So I wrote some good old VBA in Excel to connect over SSH to the mainframe and download all the data. I had one control sheet that you enter the trial number into, a button to click and away it would go downloading the data and filling out a second sheet.

Next was replacing the highlighter pen! Easy to do in Excel, work through each cell to get the action for each step, highlight the cells as if I was using a highlighter pen and paper, and extract the data for each action.

Once I had the actions, it was easy to convert from the hex codes to the steps information that was wanted. Same with locations - easy to extract and plot the chart.

Finally it was time to create a word doc - and VBA can do this nicely.

The result

All in all I spent about 4 weeks on this, and was able to generate a report ready for analysis in about 10 minutes, as opposed to 1-2 weeks. I packaged this as an add-in, so anyone could use it - just add the add-in, enter the trial number and away it went and spat out the doc.

I even gave it a cool name: Spearfish Trials Analysis Report Writers Automated Reporting Suite - or STAR WARS for short (this was a few months after Episode 1 came out after all).

I was impressed, I could save everyone time!

The reception

Everyone hated it. Simple as that. I was told there was no way it could be better than a human, and one of the top report writers challenged me to produce a better report. In 10 minutes I had one, in 4 days she had her version, and she’d made some mistakes - which made her hate it (and me) even more.

After I left, it was deleted and everyone forgot about its existence. It took me a while to figure out why - everyone was afraid. This one spreadsheet could replace almost the entire team of 7 people. You’d just need this spreadsheet and 1 person to do the analysis as the bulk of the job was manually processing reports. I could have put 6 people out of work in just 4 weeks.

Regardless of how you feel about waste in enterprises, this was 6 humans, with families who had houses, mortgages, rent, bills, who needed to eat and to live. The lives of 6 families who could have been destroyed with one spreadsheet.

Conclusion

Yes, automation is coming. Yes, technology can solve so many things. Yes, in theory if a job is replaced by automation then folks can focus their knowledge and energy onto solving bigger problems. But the reality is business is run by people who want to make as much money as possible, so automation is a way they can cut jobs and boost shareholder value, whatever that is (I’m guessing making rich folks richer).

This is not always the case, and this is a very cynical and dystopian view, but in most cases this is the truth. Even in the high-paying tech world of multi-trillion dollar companies, folks are dropped in an instant to boost share prices.

So it’s not surprising folks are worried about computers taking their jobs. If we are in the throes of an AI revolution, then maybe we need to really push the conversations around universal basic income, and taxing large corporations and billionaires to pay for it.

Installing liblab into a devcontainer using NPM

Fri, 04 Aug 2023 00:00:00 +0000

In this post we look at how to add NPM packages to a devcontainer to use in VS Code locally or in a GitHub codespace

I recently started at liblab as a Principal Developer Advocate. liblab is a tool that converts APIs into language SDKs, something I’m a big fan of as I love typesafety and the inherent documentation that comes with a good SDK. Obviously my first job was to dive into building out an API and creating an SDK from it.

Devcontainers

I love devcontainers - they allow me to create a repo that anyone can open and instantly have a fully configured environment that just works! My standard process when creating any sample code is to start with a devcontainer. In this case, I’m creating a FastAPI application, and building an SDK from that.

I start with a Python container - this is a pre-configured container that I can just spin up inside VS Code.

Once in this container I can create my API, then build the SDK using liblab.

Add liblab to the container

liblab is available as a CLI that you install via NPM. There are a number of ways you can install NPM packages in the container:

Install it manually in the postCreateCommand
Use a DOCKERFILE and install it there
Use the NPM package feature

I went for option 3 - use the NPM package feature. This avoids having to create a DOCKERFILE and configure my devcontainer.json to use it, and means I don’t have to add more to my postCreateCommand which already installs pip packages.

You can add this feature manually into the devcontainer.json file, or use the command palette:

To add manually, add the ghcr.io/devcontainers-contrib/features/npm-package:1 feature:

{
 ...
 "features": {
 "ghcr.io/devcontainers-contrib/features/npm-package:1": {}
 }
}

Once added, the feature is configured using the package and version parameters. THe package should be set to liblab, the name of the NPM package for liblab. The version parameter defaults to latest and I always want the latest liblab CLI installed, so this can be left out.

{
 ...
 "features": {
 "ghcr.io/devcontainers-contrib/features/npm-package:1": {
 "package": "liblab"
 }
 }
}

Rebuild the container

Once this is set up you can rebuild the container and the liblab SDK will be available in your terminal!

vscode ➜ /workspaces/python $ liblab
The offical liblab CLI

VERSION
 liblab/0.9.20 linux-arm64 node-v18.17.0

USAGE
 $ liblab [COMMAND]

TOPICS
 hooks Bootstrap an SDKs hooks.

COMMANDS
 build Build SDK(s) and Docs for your API.
 help Display help for liblab.
 init Create a liblab folder for housing files relevant to SDK and Doc generation.
 login Login to your account.
 logout Logout of your account.
 post-build Run extra tasks after code generation.
 pr Create a PR for all the generated SDKs.
 register Register for a new liblab account.
 reset-password Reset your password.
 token Create a new login token for CI/CD.

Learn more

Want to learn more about liblab for automating the generation of SDKs from your APIs? Get in touch.

Let's get personal: computing has won a Telly award

Thu, 01 Jun 2023 00:00:00 +0000

This year is the 44th annual Telly Awards. The Telly Awards honor excellence in video and television across all screens.

The Telly Awards was founded in 1979 to honor excellence in local, regional and cable television commercials with non-broadcast video and television programming added soon after. With the recent evolution and rise of digital video (web series, VR, 360 and beyond), the Telly Awards today also reflects and celebrates this exciting new era of the moving image on and offline.

The Telly Awards annually showcases the best work created within television and across video, for all screens. Receiving over 12,000 entries from all 50 states and 5 continents, Telly Award winners represent work from some of the most respected advertising agencies, television stations, production companies and publishers from around the world. The Telly Awards recognizes work that has been created on the behalf of a client, for a specific brand and/or company (including your own) or self-directed as a creative endeavor.

This year we entered the Build a 3D Printed Clippy Powered by ChatGPT from OpenAI, Azure Cognitive Services episode of Let’s get personal: computing, and we won a Silver award in the online talk show/interview category!

Run Azure Functions with Python on Apple Silicon

Mon, 08 May 2023 00:00:00 +0000

If you are an Apple user and you want to build Azure Functions using Python, then you’ve been out of luck. The Azure Functions core tools don’t support Python functions on Arm64. There is a workaround, and this post shows you how.

Arm64 support in Azure Functions

Apple moved to an Arm64 architecture with their Apple Silicon processors (currently M1 and M2) back in November 2020, with their entire linup now using Apple Silicon, except for the rather outdated Mac Pro. Despite this, support for running Python functions on Arm64 has not yet materialized. You can create functions with func init and func new, but when you run them, you get an error:

➜ functions-m1 func start
Found Python version 3.10.11 (python3).

Azure Functions Core Tools
Core Tools Version: 4.0.5095 Commit hash: N/A (64-bit)
Function Runtime Version: 4.16.5.20396

[2023-05-08T21:29:33.030Z] Failed to initialize worker provider for: /opt/homebrew/Cellar/azure-functions-core-tools@4/4.0.5095/workers/python
[2023-05-08T21:29:33.030Z] Microsoft.Azure.WebJobs.Script: Architecture Arm64 is not supported for language python.
[2023-05-08T21:29:33.409Z] Failed to initialize worker provider for: /opt/homebrew/Cellar/azure-functions-core-tools@4/4.0.5095/workers/python
[2023-05-08T21:29:33.409Z] Microsoft.Azure.WebJobs.Script: Architecture Arm64 is not supported for language python.
[2023-05-08T21:29:33.558Z] A host error has occurred during startup operation '2a13f8c8-f2e4-4da0-a88c-205dbd3065c2'.
[2023-05-08T21:29:33.558Z] Microsoft.Azure.WebJobs.Script: Did not find functions with language [python].
[2023-05-08T21:29:33.564Z] Failed to stop host instance '1e7799d1-a87f-404a-9bc1-858c7724aeec'.
[2023-05-08T21:29:33.564Z] Microsoft.Azure.WebJobs.Host: The host has not yet started.
Value cannot be null. (Parameter 'provider')

Microsoft originally targeted support for their Python 3.10 release, but missed this. Currently they are saying it will come with Python 3.11 support, as documented in this GitHub issue. Their suggested workaround involves changing your Terminal to always run under Rosetta 2 which feels like a bad way to do things. There is a better workaround, leveraging dev containers.

Dev containers

Dev containers are docker containers that you run and connect to from VS Code. As a user these are seamless - you can still access the file system from outside of VS Code, but internally the dev environment runs in the container, with all your dependencies installed in the container instead of on your host OS. You can read about these on the VS Code developing inside containers documentation.

Docker supports running x86 and x64 containers on Apple Silicon Macs using either QEMU emulation, or using Rosetta 2. This means you can create an x64 container and run this on your Mac, running the Azure Functions core tools inside the container! Pretty cool, hey!

Configure docker

You need docker installed, so get this installed first. Once installed, you need to configure a few things.

From the docker menu item, select Settings
From the General section, ensure Use Virtualization framework is selected
From the Features in development section, select the Beat Features and select Use Rosetta for x86/amd64 emulation on Apple Silicon
Apply these changes and docker desktop will restart.

Create the dev container

The default Azure Functions and Python3 dev container is ideal for this as it is an x86/x64 container.

Make sure docker is running.
Create a folder for your Azure Functions app and open it in VS Code (or open the one you already have created)
Ensure you have the Dev Containers extension installed. This also comes with the Remote Development extension Pack.
Open the command palette and select Dev Containers: Add Dev Container configuration files…
Select Show all definitions to see all the dev container options
Search for and select Azure Functions & Python 3
Select any other features you want. You don’t need any for Azure Functions to work.
The container files will be built, and an option will pop up to re-open the folder in the container. Select Reopen in Container
Once the repo has been re-opened in the container, open the terminal and you will see it marked as running under Rosetta
If you don’t already have an Azure Functions app, you can now create one.

Run your Azure Functions app!

vscode ➜ /workspaces/functions-arm $ func start
Found Python version 3.9.16 (python3).

Azure Functions Core Tools
Core Tools Version: 4.0.5148 Commit hash: N/A (64-bit)
Function Runtime Version: 4.17.3.20392


Functions:

 HttpTrigger: [GET,POST] http://localhost:7071/api/HttpTrigger

For detailed output, run func with --verbose flag.
[2023-05-08T22:07:35.900Z] Worker process started and initialized.
[2023-05-08T22:07:38.282Z] Host lock lease acquired by instance ID '0000000000000000000000008A10FEBD'.

Once you commit the devcontainer files to GitHub, you will also be able to open this in a Codespace and have it pre-configured with everything you need. You can also add more things to the container depending on your needs, such as extensions that are automatically loaded when the container is loaded, install things, whatever you need! Check out the the VS Code developing inside containers documentation to learn more.

Build a ChatGPT clone in .NET using OpenAI and Semantic Kernel

Tue, 11 Apr 2023 00:00:00 +0000

ChatGPT is the latest big thing, reaching over 100 million users in just 2 months. It’s being used by millions of people for all manner of tasks, and it feels like every day there are 10 new startups leveraging either ChatGPT or the large language models (LLMs) that power it.

So what if you want to build your own version of ChatGPT, or another app that leverages an LLM? Out of the box, LLMs just provide an API to send a prompt and get a response, so what tools are there to help?

One tool is Semantic Kernel, an open source .NET library from Microsoft that is designed to do a lot of the heavy lifting when it comes to interacting with LLMs.

If you prefer video, here’s a 10 minute video version of this blog post

How does ChatGPT work with an LLM?

ChatGPT allows you to have a conversation with a large language model, with ChatGPT being able to base the response to one question on the results of previous questions.

For example, if you ask it:

What is the most recent star wars movie?

It will respond something like:

The most recent Star Wars movie is “Star Wars: The Rise of Skywalker,” which was released in December 2019. It is the ninth and final installment in the Skywalker saga.

If you then ask a follow up question such as:

Who is the main character?

The response will ‘know’ that the question refers to the movie the Rise of Skywalker:

The main character in “Star Wars: The Rise of Skywalker” is Rey, played by Daisy Ridley. She is a powerful Jedi who is trying to find her place in the galaxy and confronts the evil First Order, led by Kylo Ren, played by Adam Driver. The movie also features other beloved characters from the Star Wars franchise, such as Finn, Poe Dameron, Chewbacca, and C-3PO.

LLMs, the core technology behind ChatGPT, don’t chat – they take a prompt and return a response, that’s all. To chat, they need to be guided by having access to the previous questions and responses.

ChatGPT works by doing the following:

The user enters a question, referred to as a prompt
This prompt is enriched by a system prompt that helps guide the LLM, such as ‘you are a helpful chatbot’
The prompt is sent to the LLM and the response is returned to the user
The user enters a second question
This time the prompt is enriched by the system prompt as well as the previous question and response
The prompt is sent to the LLM and the response is returned to the user

For example, when the second question is sent, the full prompt might be:

You are a helpful chatbot. You were asked ‘What is the most recent star wars movie?’ and you answered ‘The most recent Star Wars movie is “Star Wars: The Rise of Skywalker,” which was released in December 2019. It is the ninth and final installment in the Skywalker saga’. Who is the main character?

To implement your own ChatGPT you need to implement code to get the prompt from the user, enrich it with a system prompt, send it to the LLM, get the response, then enrich further questions with the previous questions and responses.

How can semantic kernel help?

Semantic Kernel provides a load of features to help you implement this:

Services to interact with LLMs from OpenAI either directly or via the Azure OpenAI service
System prompts to pre-configure the conversation
Chat history, or ‘memory’
Semantic functions that can be defined in code such as getting user input and returning the response, or defined as prompts that can be sent to the LLM
Function pipelines to chain semantic functions
Core skills such as interacting with HTTP APIs or files.

Semantic functions and pipelines

The core of semantic kernel is a pipeline that runs a series of functions. These functions either take text, return text, or both. Large language models work with text, so text is the main way to interact. The idea of the pipeline is text is passed through from function to function.

For example, for a ChatGPT clone, you could have a pipeline that gets input from the user, sends it to the LLM with the previous questions and responses, then send the response back to the user.

For a sample project, check out github.com/jimbobbennett/console-gpt.

In this code, there is a ConsoleGPTService that has a pipeline:

ISKFunction[] pipeline = {_speechSkill["Listen"], _chatSkill["Prompt"], _speechSkill["Respond"]};
await _semanticKernel.RunAsync(pipeline);

This pipeline runs 3 semantic functions, routing the output of one into the input of the other. Semantic functions are C# functions that are marked with an attribute and loaded into the pipeline.

The Listen function in this code is defined in the ConsoleSkill and gets data from the console:

[SKFunction("Get console input.")]
[SKFunctionName("Listen")]
public Task<string> Listen(SKContext context)
{
 return Task.Run(() => Console.ReadLine());
}

This returns what the used inputs, and this is passed to the Prompt semantic function in the ChatSkill:

[SKFunction("Send a prompt to the LLM.")]
[SKFunctionName("Prompt")]
public async Task<string> Prompt(string prompt)
{
 _chatHistory.AddUserMessage(prompt);
 var reply = await _chatCompletion.GenerateMessageAsync(_chatHistory, _chatRequestSettings);

 // Add the interaction to the chat history.
 _chatHistory.AddAssistantMessage(reply);

 return reply;
}

This uses the AddOpenAIChatCompletionService, a service provided by the semantic kernel to interact with OpenAI. This service is a chat service, so manages history - when you call it you pass the complete chat history including a system prompt that sets up the conversation, and all the prompts and responses.

The output of this is then passed to a Respond semantic function to write it to the console:

[SKFunction("Write a response to the console.")]
[SKFunctionName("Respond")]
public Task<string> Respond(string message, SKContext context)
{
 return Task.Run(() => Console.WriteLine(message));
}

By using semantic functions you can quickly swap out functions by implementing the same signature in multiple places and bringing the one you want into the semantic kernel. You can also construct pipelines on the fly, adding sections if you need.

Semantic functions can also be created using prompts rather than code. For example:

var poemPrompt = """
 Take this "{{$INPUT}}" and convert it to a poem in iambic pentameter.
 """;

_poemFunction = _semanticKernel.CreateSemanticFunction(poemPrompt);

You can then add this to the pipeline, and the output from the previous function in the pipeline is passed to the {{$INPUT}} variable in the prompt. In this case using the LLM to convert to a poem.

ISKFunction[] pipeline = {_speechSkill["Listen"], _chatSkill["Prompt"], _poemFunction, _speechSkill["Respond"]};
await _semanticKernel.RunAsync(pipeline);

This means for the prompt:

What is the most recent star wars movie?

You might get this poem:

The Force of Fate has brought us here,

To see the Rise of Skywalker near,

The ninth and final part of this tale,

Released in December, ’twas no fail.

Star Wars, the movie, we all know,

The saga of the Skywalkers show,

The Force will live on, we can see,

In this movie, ’tis our destiny.

Get the code

You can get the code on my console-gpt repo on GitHub.

Learn more

Here are some more resources for Semantic Kernel

An intro to generative AI created by generative AI!

Fri, 31 Mar 2023 00:00:00 +0000

Generative AI is the new hotness - using AI models trained on a huge amount of data that can generate new data. This includes generating text or images. I decided to create a video giving an explanation of generative AI using - you guessed it, generative AI!

Check out the video here:

Creating this video

To create this video I needed a few things:

An image of a person to use for the presenter
A script
A way to convert the script to spoken audio
A way to make the image move in time with the audio
A YouTube description

My plan was to do all of this using AI.

Create the image

To create the image, I decided to use DALL-E-2, the latest image generation model from OpenAI. This model takes a prompt and generates an image. You can either use it through their website at labs.openai.com, or you can generate images using their API.

When creating a prompt you need to be as descriptive as possible so that the AI can generate the image. The website generates 4 images for each prompt, all different.

This is the prompt I used:

A hyper realistic bust upwards portrait of a classically beautiful mid-20s woman of indeterminate race with rainbow hair, who looks like a cross between a CEO and an opera singer and is posed facing the camera. She is wearing a colorful dress and is standing in front of a plain black backdrop.

These are the 4 images created:

I picked the one with the full head as I wanted to animate the full face.

This took a few iterations to get a prompt that created images I liked, such as age, outfit and style of dress. Most of the images were of the right quality, but some had a few quirks. For example, the eyes in this image were not well generated:

DALL-E creates square images, at a default of 1024x1024, so I knew I would need to add extra sizing on the sides or top and bottom to get the right aspect ratio. THis is why I added standing in front of a plain black backdrop to the prompt, so I could fill out the extra space with a single color.

Once generated, I downloaded the image, and then resized it to a higher resolution using the ML super resolution feature of Pixelmator Pro.

Create the script

Next up was something for the generated AI character to say. For this I decided to use ChatGPT with the GPT4 model. I decided to sign up for this at US$20 a month rather than wait, so will see if it is a worthwhile investment.

ChatGPT generates text for you based of textual prompts, and has a ‘memory’ to remember your conversation and use this as the input to the next prompts, allowing you to have a conversation-like discussion.

This is the prompt I used:

You are Ganya, the CEO of an AI company and your company is called Alvie. Write a 5 minute talk that covers an introduction to generative AI and how it can benefit businesses. This talk needs to be entertaining with a couple of jokes, but also informative. The audience for this talk is a group of students who don’t have strong technical knowledge.

The name Ganya is a unisex name used with Hebrew origins, and means clever. Alvie is a English female name, and is short for Alvina. It is a name which means wise friend. I picked these names at random from a baby name site listing names that mean intelligent.

This is the output it generated:

Ladies and gentlemen, future tech wizards, and brilliant entrepreneurs, thank you for inviting me to speak today. I am Ganya, the CEO of Alvie, and I’ll be your digital tour guide on this wild journey through the fascinating world of generative AI. So, buckle up, because we’re about to embark on an adventure that’s more thrilling than binge-watching a season of your favorite sci-fi show!

Now, you might be wondering, “What the heck is generative AI?” Well, you know how your phone’s predictive text often tries to guess what you’re typing before you even finish typing it? Yeah, that’s a tiny taste of generative AI! It’s like a mind-reading superpower, but for machines. And it’s not only limited to text, it can be used for images, music, and even entire virtual worlds.

You see, generative AI is the cool cousin of traditional AI. Instead of being the nerdy, know-it-all who’s good at math, generative AI is the creative type who spends their time painting, writing poetry, or designing websites. It’s the kind of AI that dares to dream, and it’s revolutionizing the way we think about technology.

But let’s take a step back and look at the big picture. Generative AI is like a chef, and data is its secret ingredient. It takes massive amounts of data, mixes it all up, and uses this delicious concoction to create brand-new content, whether that’s text, images, or music. Essentially, it’s a digital Picasso with a dash of Shakespeare and a sprinkle of Mozart.

Now, I know what you’re thinking: “That’s all great, Ganya, but how can generative AI help my business? I’m not running an art gallery or a concert hall!” Well, fear not! Generative AI is a versatile beast and can benefit businesses in many ways.

For example, take customer service. We all know how frustrating it can be to deal with a grumpy customer service rep who clearly didn’t have their morning coffee. Enter generative AI: it can be used to create friendly, informative, and highly responsive chatbots that can handle thousands of customer inquiries at once, without ever needing a caffeine fix. That’s what I call efficiency!

Speaking of efficiency, generative AI can also help companies optimize their workflow. Imagine having a smart algorithm that can analyze complex data sets and automatically generate insightful reports for your team. It’s like having an intern who never gets tired, never complains, and doesn’t need a salary. The only downside? You can’t send it out to fetch you coffee, but hey, nobody’s perfect!

Now, I can’t let you leave without a joke or two, can I? So, here’s one for you: Why did the generative AI go to therapy? Because it had deep learning issues! (Pause for laughter) Okay, okay, I promise I won’t quit my day job.

But in all seriousness, folks, generative AI is an incredible tool that can transform the way we do business. It can help companies save time, money, and resources, while also enabling us to create new, innovative products and services. As we embark on this exciting journey together, let’s remember that the future of AI is not just about cold, hard numbers; it’s about tapping into the boundless creativity that lies within each and every one of us.

Thank you for your time, and I hope this little chat has sparked your imagination and inspired you to explore the fantastic world of generative AI!

Convert the script to spoken audio

There are a number of tools that can take text and create spoken audio. The best ones use AI to make the voice sound lifelike with changes in timing and intonation that mimic how a real person would speak. As a Microsoft person I could have used the Cognitive services speech service but this would have meant writing code, and I couldn’t be bothered…

Instead I used Eleven Labs which has a free tier for up to 10,000 characters per month. You can select from a range of different voices, paste in your text and it creates the audio as an MP3 file. There is a limit of 2,500 characters per conversion, so I had to do it in 2 chunks as my text was 3,335 characters long, then use Camtasia to stitch the audio files together.

Here is the generated audio:

Create a video speaking the audio

The next step was to create the video by mashing up the audio and the image, and animating the image to speak the words. There are a number of tools that could contribute to this, such as Make it talk from Adobe research, but I wanted a solution that I didn’t need to run myself.

I ended up using D-ID which is not cheap! They have a 14 day free tier for up to 5 minutes of video, but beyond that it gets very expensive. It works really well though! You can either use one of their avatars, or upload an image yourself. Once they have the image, you provide either the script and it will generate the audio for you (using the Microsoft Cognitive services under the hood), or upload the audio.

It then allows you to download the final video, and because I had a square image it sized it to 1280x1280. I then open up Camtasia one more time to create 2 videos with this - one at 1920x1080 for YouTube, and one at 1080x1920 for Instagram, filling the extra space around the video with the background color which was slightly off-black.

Create a YouTube video

Now I had my video, it was time to upload to YouTube. I uploaded the video and knocked up a simple thumbnail with text for the video name manually. Tools like DALL-E are not good for adding text to images, so something to research is a tool for this.

I then needed a description, so it was time to go back to ChatGPT! The prompt I used was:

Generate a YouTube description for a video with the following text:

<the script>

I then added the complete script to the prompt (I cut it off here for brevity). ChatGPT then generated the following:

Title: Exploring the World of Generative AI with Alvie CEO Ganya

Description:

🚀 Join Ganya, the CEO of Alvie, as she takes you on an exciting and informative journey through the world of generative AI! In this video, Ganya breaks down what generative AI is, how it works, and its potential to revolutionize the way we do business. From creative applications like art, music, and writing to practical uses in customer service and workflow optimization, generative AI offers a world of possibilities.

🤖 Key Points: 0:00 - Introduction and Welcome 1:05 - What is Generative AI? 2:38 - How Generative AI Works: Data and Creativity 4:15 - Benefits of Generative AI for Businesses 6:45 - The Future of AI and Creativity 8:30 - Jokes and Closing Remarks

🔮 Don’t miss this opportunity to learn about the fantastic world of generative AI and its potential to unlock boundless creativity in technology and business. Hit the subscribe button to stay updated on the latest advancements and insights from Alvie!

#GenerativeAI #Alvie #AIinBusiness

Pretty good stuff! I removed the #Alvie tag as this makes no sense as Alvie doesn’t exist, and had to re-do the time stamps for the chapter markers as ChatGPT has no way of knowing the timings. But otherwise it was perfect!

I added a disclaimer at the start of the description to say this was AI generated, and listed the tools.

And I was done! Video created and uploaded all powered by generative AI!

What’s next?

My next job is to try to automate all this into a tool so I can give it a prompt and have it create and upload the full video without any manual intervention.

Embedding audio files in a Hugo site

Fri, 31 Mar 2023 00:00:00 +0000

I was writing a post today and wanted to embed an mp3 file of some text to speech output. Hugo doesn’t support this natively using shortcodes, so I needed a way to add these, ideally without adding any HTML.

To do this, I needed to create my own shortcode implementation. As it turns out, based on the Hugo shortcode guide, these are not to complicated.

Create the shortcode

Shortcodes live in a folder called shortcodes in your layout folder and are implemented as HTML files, named as <shortcode>.html. For example, if you wanted to create a shortcode called audio you would create the file layout/shortcodes/audio.html.

Shortcodes are snippets of HTML that can be passed named parameters. The HTML for an HTML audio player is:

<audio controls preload="auto">
 <source src="file.mp3">
</audio>

Shortcodes can also be parameterized with name parameters that you can get using the {{ .Get "name" }} method, passing the name of the parameter. For the audio shortcode, I need to pass in the audio file, so this can be a parameter. This is set in the src field, so I called this parameter src:

<audio controls preload="auto">
 <source src="{{ .Get "src" }}">
</audio>

Done! This is my entire shortcode.

Use the shortcode

Once my shortcode was written, it was easy to use. I added the mp3 file I want to play to the folder for my blog post, and added the shortcode tag in the markdown file for the post:

{{< audio src="intro-to-generative-ai.m4a" >}}

Once done, the audio player appears on my page:

Use this yourself

If you want to use this shortcode, you can find it on the GitHub repo for this site.

A 2-day open source bootcamp in a box

Thu, 30 Mar 2023 00:00:00 +0000

I’m excited to announce the publication of a 2-day open source bootcamp in a box!

This boot camp was put together by Sam Wronski, Austin Akers and I, with help and support from GitHub, GitHub campus experts, and others from the open source community.

Goals

The goal of this bootcamp is to take attendees from a very basic to no understanding of GitHub on a journey to not only being comfortable with collaborating on GitHub, but being ready to actively contributing to public open source projects, or to collaborate on code in the workplace. This bootcamp is structured as an in-person event.

The code the attendees will be working on is relatively basic JavaScript, and attendees without JavaScript knowledge but with some knowledge of programming should still be able to successfully complete this bootcamp.

Running this event

This GitHub repo contains the full instructions on how to run this event, with guides for the day, slide decks, a sample project to work on with full instructions, videos of the sessions, and plenty of suggestions on how to make this day a success. This bootcamp is fully open source and MIT licensed, so you can take it and run it at your own event without cost or attribution.

When we have run this event in the past (such as at the In-person Student Open Source Bootcamp, Hacktoberfest in-person open source bootcamp, and the Fall in love with Open Source bootcamp, all hosted at the Microsoft Reactor in Redmond). The attendees have all given feedback that networking is important, therefore this event includes opportunities for attendees to work together on code, be involved in networking

Let me know what you think!

If this bootcamp is of use to you, please let me know! If you have any questions or suggestions for improvements, please start a discussion in the GitHub org.

Install the Azure CLI on a Raspberry Pi

Wed, 08 Mar 2023 00:00:00 +0000

I do a lot with Raspberry Pis, and sometimes I want all my tools installed in one place so I can use my Pi for everything, rather than flipping back to my Mac.

One thing I use a lot is Azure - funny really as I work for Microsoft! I often use the Azure portal as I prefer UIs to CLIs, but when working on a Pi I regularly use Raspberry Pi OS Lite, so don’t have a browser to use as I’m always in the terminal or VS Code. So I needed the Azure CLI on my Pi.

Hardware

I’m currently doing everything on a Raspberry Pi 4 as I have a few of them (will trade one for a Lamborghini - no low ballers, I know what I have), so have one of these set up with Raspberry Pi OS Lite 32-bit. I’ve not tested this on 64-bit, but I’m guessing it should work.

Pre-requisites

To install the CLI, you can’t just use the apt package - this only currently supports x86, not armhf. Instead it needs to be installed from an install script that installs a raw Python CLI. This has a few dependencies:

Run the following to ensure everything is installed:

sudo apt install libffi-dev python3-dev python3-pip openssl

Install the Azure CLI

Once the pre-requisites are installed, you can use a handy script from Microsoft to install the CLI:

curl -L https://aka.ms/InstallAzureCli | bash

Run this with all the defaults.

This adds the az command to the /home/pi/bin/ folder, and gives you an option to add this to your path, which you should accept. You’ll then need to manually restart your shell or run exec -l $SHELL to restart it.

Once installed, you can log in with az login. It’s smart enough to realize you don’t have a web browser, and take you through the device login path, giving you a code. Head to https://microsoft.com/devicelogin and enter the code given to get logged in!

Run Appwrite on a Raspberry Pi

Thu, 02 Mar 2023 00:00:00 +0000

TL;DR - use Raspberry Pi OS 64-bit if you want to run Appwrite on a Pi

I’ve started working on a personal project, so thought it might be fun to give Appwrite a spin. Appwrite is an open-source app backend for web and mobile projects, you can think of it as a competitor to Firebase.

Rather than use a hosted version, or try to run it on Azure and burn through my credits, I decided to run it on a local server. I have a few Raspberry Pi’s kicking round, so thought Id try running it on there.

Hardware

Appwrite claims to run on as little as 1 CPU and 2GB of RAM, so a Pi 4 with 4GB should be more than enough. I have one spare, so set it up.

Raspberry Pi OS

The first thing to note is that Appwrite does run on Arm, but only Arm64. The ‘default’ Raspberry Pi OS is the 32-bit version, and Appwrite will not run on this. Instead, when you set up your SD card, you need to use Raspberry Pi OS 64-bit. In my case I’m using the lite version as I want to run this as a headless server.

I installed this on an SD card, then booted up my Pi 4.

Configuring the software

Appwrite runs as a docker container, so the first thing to do is install docker. The convenience script from docker works perfectly on the Pi:

curl -fsSL https://get.docker.com -o get-docker.sh
sudo sh ./get-docker.sh

Takes a while, but this sets up docker. I then like to add the current user to the docker group to save sudoing all the things.

sudo usermod -aG docker $USER

Now docker is installed, you can install Appwrite. Appwrite has 2 ways to do this - a quick install and a manual install. I decided to do the manual install in case I needed to configure things later.

You start by creating a folder to run from on your Pi, then downloading a docker compose and .env file:

mkdir appwrite
cd appwrite

curl -o docker-compose.yaml https://appwrite.io/install/compose
curl -o .env https://appwrite.io/install/env

From here, you can build the container and start it up:

docker compose up -d

This runs the container detached - as in it runs in the background, returning to the current session. If you close your shell it will stay running.

Once I had this running, I could then open Appwrite by accessing my Pi from a browser. Appwrite runs on port 80, so listens to default HTTP requests on the Pi. From there, create a new account to access the server.

Control Microsoft Teams with a Stream Deck

Thu, 02 Feb 2023 00:00:00 +0000

Already have a Stream Deck and just want to get it working with Teams? Jump to the instructions to set this up

Like a lot of folks, I spend time in Microsoft Teams. It’s where I chat to my team and others at Microsoft, and it’s where I take most of my meetings.

Although Teams has a lot of good features, the UI can sometimes be a pain to use in meetings - especially as a Microsoft employee where we get to try the dogfood versions and things move all the time. There have been loads of occasions where I have tried to give a reaction and put up my hand by mistake, or tried to mute and left the call. WHat I really need is a tool to allow me to control teams without using the UI.

Makers have worked on hacks for this. For example, the key combination ctrl+shift+m toggles muting on teams. Add a big red button that simulates this key press and you have a quick way to mute or unmute. Here’s one example of this from Jen Fox on Hackster.

For those with a less maker mindset, there’s now a new way to do this - using a Stream deck!

If you’ve not come across a Stream Deck before, its a neat little gadget that comes in different sizes and has an array of LCD buttons. You can set these buttons up to do things like control apps, or make REST requests, and as each button is an LCD screen you can fully customize what is displayed. For example, you can use one of these to control apps like OBS to switch scenes, or start streaming to Twitch.

There’s a whole Stream Deck store with plugins to extend the capabilities by controlling other apps or hardware (such as Phillips Hue lights). A Teams plugin has just been released!

Set up your Stream Deck for Teams

To set up the Stream Deck, you need an API key for Teams to allow external apps to control it - makes sense really as Teams gives access to confidential work stuff, so you don’t want any app to be able to get access. Then you install the plug in, and set it up.

Get an API key

To get an API key:

Select the ellipses from the top-right corner of Teams, then select Settings
Select the Privacy tab, then scroll down to the Third-party app API section and select Manage API
From the API settings, copy the API token. Use the Generate button to generate a new API key if needed. And yes, the UI is terrible and the key wraps out of the text box… 🤷

Now you have your key, you can set up your Stream Deck!

Set up the Stream Deck

Teams is managed from a plugin from the Stream Deck store.

Open the Stream Deck store from the menu/taskbar icon
Search for Teams in the store, and install the Microsoft Teams plugin with the install button

You might also see this on the Store discover page - at the time of writing it’s the first item there.
Once installed, configure your Stream Deck using the Teams options. I did mine by creating a new profile that is activated when Teams is active.
Once your profile is ready, select it then add buttons by dragging them from the Microsoft Teams group in the menu. After adding each button, you will need to set the API token field to the API key you copied from Teams.

Done! You can now control Teams using your Stream Deck.

Format Rust code with cargo format

Thu, 26 Jan 2023 00:00:00 +0000

This blog post was a fun one to write - cos I didn’t! I actually learned about cargo fmt on a recent Crack the code interview with Rust live stream and decided a quick blog post would be fun.

So I launched ChatGPT and asked it to write this post! This is one of 2 posts written this way, so you will notice how similar this is to my cargo clippy post.

Rust is a popular programming language known for its safety and performance. One of the great things about Rust is the abundance of tools available to make development easier and more efficient. One such tool is cargo fmt, which is a command line utility for formatting Rust code. In this blog post, we’ll show you how to use cargo fmt to make your Rust code more readable and consistent.

First, make sure that you have Rust and Cargo installed on your machine. If you don’t, you can download and install them from the official Rust website. Once you have Rust and Cargo set up, you can use the cargo fmt command to format your code.

To format a specific file, navigate to the root directory of your project in the command line and run the following command:

cargo fmt --file=path/to/file.rs

This will format the file located at path/to/file.rs.

If you want to format all the files in your project, navigate to the root directory of your project in the command line and run the following command:

cargo fmt

This will format all the files in your project.

You can also format all the files in your project and check for formatting errors by running the following command:

cargo fmt --check

This will check for formatting errors and display them in the command line. If there are no errors, the command will exit with a 0 status code, otherwise it will exit with a non-zero status code.

In addition to formatting files, cargo fmt also provides options to configure the formatting behavior. For example, you can configure the number of spaces used for indentation or the maximum line width. You can find more information about these options in the official Cargo documentation.

In conclusion, cargo fmt is a powerful tool that can help you to make your Rust code more readable and consistent. It’s easy to use and provides a lot of options to customize the formatting behavior. By using cargo fmt, you can spend less time worrying about formatting and more time focusing on writing great code.

Lint Rust code with cargo clippy

Thu, 26 Jan 2023 00:00:00 +0000

This blog post was a fun one to write - cos I didn’t! I actually learned about cargo clippy on a recent Crack the code interview with Rust live stream and decided a quick blog post would be fun.

So I launched ChatGPT and asked it to write this post! This is one of 2 posts written this way, so you will notice how similar this is to my cargo fmt post.

Rust is a powerful programming language known for its safety and performance. One of the great things about Rust is the abundance of tools available to make development easier and more efficient. One such tool is cargo clippy, which is a command-line utility for linting Rust code. In this blog post, we’ll show you how to use cargo clippy to identify and fix potential errors in your Rust code.

To lint a specific file, navigate to the root directory of your project in the command line and run the following command:

cargo clippy --file=path/to/file.rs

This will lint the file located at path/to/file.rs.

If you want to lint all the files in your project, navigate to the root directory of your project in the command line and run the following command:

cargo clippy

This will lint all the files in your project.

Cargo clippy will provide suggestions and warnings for potential errors in your code, such as unused variables, unnecessary operations, and more. For example, it will suggest more efficient ways of writing your code and point out common pitfalls that could lead to runtime errors.

You can also configure cargo clippy to check for specific types of errors. For example, you can use the -A flag to check for specific lints or use -W flag to check for warnings. You can find more information about these options in the official Cargo Clippy documentation.

In addition to linting your code, cargo clippy also provides options to customize its behavior. For example, you can configure the level of verbosity or the maximum number of suggestions to display. You can find more information about these options in the official Cargo Clippy documentation.

In conclusion, cargo clippy is a powerful tool that can help you to identify and fix potential errors in your Rust code. It’s easy to use and provides a lot of options to customize its behavior. By using cargo clippy, you can spend less time worrying about errors and more time focusing on writing great code.

Run an Ubuntu VM on your Mac with a single command

Tue, 24 Jan 2023 00:00:00 +0000

I recently needed to run Linux so I could test out what Python version was installed, and how to upgrade it for some documentation I’m creating.

I don’t have a spare machine to set up as a Linux box, and didn’t want to use a Raspberry Pi, I wanted a clean Ubuntu install.

My original thought was to use VMWare Fusion Player, and spin up a VM, but things got a bit tricky - I installed it, created a new Ubuntu VM, and it just didn’t work. It didn’t have any install media, and just failed to boot. I’m not sure why, I just assumed ‘Create an Ubuntu VM’ would just work…

So digging for an ISO, I cam across another method to install - Multipass from Canonical. No, this isn’t cheap movie tickets, instead its a free way to manage VMs using the hypervisor built into your OS - Hyper-V on Windows, QEMU and HyperKit on macOS and LXD on Linux.

Installing Multipass

Multipass was easy to install on my Mac - it can be installed via homebrew:

brew install --cask multipass

==> Downloading https://github.com/canonical/multipass/releases/download/v1.11.0
==> Downloading from https://objects.githubusercontent.com/github-production-rel
######################################################################## 100.0%
==> Installing Cask multipass
==> Running installer for multipass; your password may be necessary.
Package installers may write to any location; options such as `--appdir` are ignored.
Password:
installer: Package name is multipass
installer: Installing at base path /
installer: The install was successful.
🍺 multipass was successfully installed!

I had to enter my password (which makes sense as this installs some OS level stuff), but in a few seconds it was installed.

Creating a VM

Creating a new VM is easy - just one command to create and launch it:

multipass launch

This takes a loooooooong time the first time as it needs to download an image and create the new VM. Once the VM is created it will launched given a two-word name.

➜ ~ multipass launch
Launched: disarming-woodcock

You can get more info on the VM using the info command with the instance name:

➜ ~ multipass info disarming-woodcock

Name: disarming-woodcock
State: Running
IPv4: 192.168.64.2
Release: Ubuntu 22.04.1 LTS
Image hash: 8593ce1c6bbd (Ubuntu 22.04 LTS)
CPU(s): 1
Load: 0.27 0.17 0.07
Disk usage: 1.4GiB out of 4.7GiB
Memory usage: 149.2MiB out of 962.7MiB
Mounts: --

Once the VM is running, you can log in to it using the shell command with the instance name:

multipass shell disarming-woodcock

Done! I now have an Ubuntu VM ready to play with.

Subscribe to messages from an iRobot Create3 using ROS

Thu, 22 Sep 2022 00:00:00 +0000

In my last post I showed how I connected a Raspberry Pi to an iRobot Crete3 robot. In this post I show how to subscribe to messages sent by the robot to the Pi using rclpy, the ROS Python library.

One downside to ROS is that the documentation needs some love. It covers in great detail how to do things, but not the why, so anything I write in this post is based off my understanding from my couple of days working with ROS, and could be completely wrong! If so, please correct me in the comments.

ROS2 applications

ROS applications are based around a graph of nodes that can publish and subscribe to messages on different topics. Each node can be a separate process, can run on a separate machine connected to the network, or you can have multiple nodes running in the same process.

The robot is a node that published messages on various topics, and can listen to messages on other topics. If you read the last post, you will know that the robot has a range of topics that you can see by running ros2 topic list:

/battery_state
/cmd_audio
/cmd_lightring
/cmd_vel
/dock
/hazard_detection
/imu
/interface_buttons
/ir_intensity
/ir_opcode
/kidnap_status
/mobility_monitor/transition_event
/mouse
/odom
/parameter_events
/robot_state/transition_event
/rosout
/slip_status
/static_transform/transition_event
/stop_status
/tf
/tf_static
/wheel_status
/wheel_ticks
/wheel_vels

The majority of these topics contain sensor data published by the robot with the exception of the 3 cmd topics - cmd_audio, cmd_lightring and cmd_vel. You can publish data to these topics to play audio, configure the light ring, or move the robot.

Topics are designed to have continuous streams of data. For example if you subscribe to the /battery_state topic, you will get a continuous stream of battery data updated every few seconds.

Robots can also have services and actions. These are abstractions that make it easier to program a robot. Services are designed for quick calls to do calculations or get states, maybe to query a sensor or take advantage of onboard processing such as a built-in image processor. Actions are commands you can send to a robot to do something, for example with the Create3 there are actions to dock and undock, so drive the robot to the docking station and connect it to recharge, or drive off the docking station.

You can test the actions using the following run from your Raspberry Pi.

Run this command to undock the robot:

ros2 action send_goal /undock irobot_create_msgs/action/Undock "{}"

Then run this command to re-dock the robot:

ros2 action send_goal /dock irobot_create_msgs/action/DockServo "{}"

All the topics, services and actions are documented in the iRobot ROS2 API docs.

Subscribing to messages using rclpy

rclpy is the ROS Python library and has all the functionality you need to interact with a robot. But there is a catch - you can’t just create a Python script and use rclpy, you have to build a ROS2 application inside a ROS2 workspace.

ROS2 workspaces

ROS applications are compiled inside a workspace - a folder with a predefined structure containing your code. You write your code, compile a ROS application using a tool called colcon, configure your environment by sourcing a file created by the build, then run your code using ROS.

There is a core ROS workspace provided by the install, known as an underlay, and you can have multiple versions installed. You then stack additional workspaces on top with your own code, called overlays. No, I still don’t understand this and I can’t find good documentation to explain why.

To use Python to developer ROS applications, you need to create a Python package inside your workspace, and this package contains the code for your executable. This is similar to creating a pip package, with setup.py and setup.cfg files, but with the addition of a package.xml file to help colcon build the package.

You can find an example as a template repo on my GitHub at github.com/jimbobbennett/irobot_create3_ros_workspace. You can use this as a template to create your own workspace.

Structure of the workspace

You can see the structure detailed blow by creating a new repo using the github.com/jimbobbennett/irobot_create3_ros_workspace template repo.

The workspace consists of a src folder where the source code for your application lives. This src folder contains the Python package as a folder. This package folder contains setup.py and setup.cfg files. These files define any extra pip packages needed by your code, the location of the package.xml file needed by colcon, and the entry point for the ROS application.

The package.xml file details the ROS package, including dependencies on other packages. These include the ROS packages that define message formats. For example:

<depend>rclpy</depend>
<depend>geometry_msgs</depend>
<depend>sensor_msgs</depend>
<depend>irobot_create_msgs</depend>

This dependency list shows a dependency on the rclpy ROS python package, as well as the packages for the generic geometry messages used to control a robot (geometry_msgs), generic sensor messages used to gather standard sensor data such as battery state (sensor_msgs), and the specific messages for the iRobot Create3 (irobot_create_msgs).

Inside the package folder is a folder containing your code, along with a resource folder with a single file with the same name as the folder containing your code. No idea why, but it seems to be needed…

The code folder is a Python package, and needs an __init__.py file, though this can be empty. Inside the code folder is another folder with your actual package code, again containing an __init__.py file that can be empty, and a code file that contains your main method, along with any other code files you need. Your setup.py needs to point to this main method as its entry point.

Subscribing to messages

If you look at the template repo, particularly the control_robot.py file you will see an example main method.

This method starts by initializing the ROS application with rclpy.init(args=args), passing any args sent to the application. It then creates a subscriber, in this case a subscriber to the battery state to get the percentage charge in the robots battery.

Subscribers are ROS nodes - ROS applications are graphs of nodes that publish and subscribe messages, and your application can contain one oe more nodes, and nodes can publish and subscribe to as many topics as needed. In the template repo, the node in question is a battery state subscriber, implemented in the BatteryStateSubscriber class:

class BatteryStateSubscriber(Node):

All nodes need to derive from the rclpy.node.Node base class, and this has functionality to manage subscriptions, logging, execution and more.

In the __init__ method, the node creates a subscription to the battery state:

self.subscription_battery_state = self.create_subscription(
 BatteryState,
 '/battery_state',
 self.get_battery_state,
 qos_profile)

The create_subscription method on the base Node class takes a number of options:

BatteryState - This is the message type. Messages can be raw strings, or typed. The sensor_msgs, sensor_msgs and irobot_create_msgs packages described earlier contain the types for these messages. In this case, we want battery information, so the type is sensor_msgs.msg.BatteryState, a ROS defined message for battery states.
'/battery_state' - This is the topic name used by the robot to publish the battery state
self.get_battery_state - This is the callback method called whenever a message is received by the subscription
qos_profile - This is the quality of service profile defined by the subscriber. This is covered below.

Once this subscription is created, every time a message is published on this topic, the callback method is called. The callback method has this signature:

def get_battery_state(self, msg: BatteryState):

The msg passed is an instance of sensor_msgs.msg.BatteryState, so the properties on this message can be queried to get the state of the battery. For example:

self.get_logger().info(
 'Battery percentage:{:.0%}'.format(msg.percentage))

uses the logger on the node to log out the current percentage charge on the battery.

[INFO] [1663974929.958794823] [battery_state_subscriber]: Subscribing to battery state
[INFO] [1663974930.072528455] [battery_state_subscriber]: Subscribed!
[INFO] [1663974931.658984386] [battery_state_subscriber]: Battery percentage:98%

Quality of service

ROS ia based around sending messages to topics, and can be considered to be a message broker. Like all message brokers, there is a quality of service (QoS) that defines when a topic can be considered stale, how messages are stored if a subscriber is offline, things like that. You can read more in the ROS quality of service documentation, but the important point is that the subscriber must subscribe with the same quality of service settings as the publisher or no messages will be received.

You can find the QoS settings from a topic using this command:

ros2 topic info -v <topic>

Where <topic> is the name of the topic to query. In the case of the /battery_state topic, the results of this call are:

Type: sensor_msgs/msg/BatteryState
Publisher count: 1
Node name: robot_state
Node namespace: /
Topic type: sensor_msgs/msg/BatteryState
Endpoint type: PUBLISHER
GID: 01.0f.34.90.5c.09.ad.d2.01.00.00.00.00.01.4f.03.00.00.00.00.00.00.00.00
QoS profile:
Reliability: BEST_EFFORT
Durability: VOLATILE
Lifespan: 9223372036854775807 nanoseconds
Deadline: 9223372036854775807 nanoseconds
Liveliness: AUTOMATIC
Liveliness lease duration: 9223372036854775807 nanoseconds
Subscription count: 0

The relevant details are the QoS profile, particularly the Reliability, Durability, and the Liveliness. To subscribe to these messages, the subscriber needs to create a QoSProfile object with the same settings, and pass this to the create_subscription call:

from rclpy.qos import QoSProfile, ReliabilityPolicy, LivelinessPolicy, DurabilityPolicy

qos_profile = QoSProfile(
 reliability=ReliabilityPolicy.BEST_EFFORT,
 liveliness=LivelinessPolicy.AUTOMATIC,
 durability=DurabilityPolicy.VOLATILE,
 depth=1
 )

self.subscription_battery_state = self.create_subscription(
 BatteryState,
 '/battery_state',
 self.get_battery_state,
 qos_profile)

If you subscribe to a ROS topic and don’t receive any messages, check your QoS settings

Running the ROS application

Once you have your ROS workspace with your code, you can run it.

The first step is to build the workspace:

colcon build

This command builds the package and creates 3 directories:

build - This contains the compiled Python package
install - This contains the application that can be installed and run on another device
log - This contains the build logs

To run the code, first you need to source an install script to set the environment:

source ./install/local_setup.sh

Finally run the ROS package:

ros2 run <package name> <application>

For example, if you use the template repo mentioned earlier in this post, you can run with:

ros2 run irobot_create3_example_py control_robot

This will run the control_robot module in the irobot_create3_example_py package, subscribing to the battery state and printing it to the console:

ubuntu@ubuntu:~/irobot_create3_ros_workspace$ ros2 run irobot_create3_example_py control_robot
[INFO] [1663974929.958794823] [battery_state_subscriber]: Subscribing to battery state
[INFO] [1663974930.072528455] [battery_state_subscriber]: Subscribed!

Connect a Raspberry Pi to an iRobot Create 3

Wed, 21 Sep 2022 00:00:00 +0000

iRobot, the makers of robotic vacuum cleaners also make educational robots designed for learning how to do robotics programming. I recently managed to get my hands on one of their new Create3 robots as part of the Microsoft Global Hackathon, one of the cool perks of working for Microsoft.

These robots are the same as the vacuum cleaners, except without the vacuum part. They have all the same sensors, with extra connectivity for you to program yourself. The beginner option is to program them in Python using their browser-based tooling that connects using bluetooth, but I wanted to do something more advanced and learn a bit about ROS!

ROS

ROS, short for Robot Operating System is, as the name suggests, an ‘operating system" for robots. Despite the name, it’s not so much an operating system, more a set of software libraries and tools for building and controlling robots that runs on top of Windows or Linux. It’s fully open source and free to use.

The downside to ROS is that the documentation needs some love. It covers in great detail how to do things, but not the why, so anything I write in this post is based off my understanding from my couple of days working with ROS, and could be completely wrong! If so, please correct me in the comments.

ROS has a concept of a graph of nodes that connect to each other, nodes can be actuators, sensors, or parts of the control code. Nodes communicate by sending messages on named topics to any other node that is listening.

The iRobot Create3 runs ROS2, the current version of ROS. ROS has version names for the minor versions, and my Create3 runs Galactic Geochelone 🤷. The Create3 can connect to a compute board running ROS that can be programmed to receive messages from the sensors on the Create3, or send messages to the actuators. The compute board can be a Raspberry Pi, a NVIDIA Jetson Nano, or an NXP NavQ+. This compute board can then connect to additional sensors or actuators (for example LIDAR or a camera), add control logic for the robot, and connect to the internet such as to IoT or AI services.

In my case, I want to connect a Raspberry Pi 4. The goal is to start by sending status data to an IoT Central dashboard, then expand to add a camera and code to detect and find objects using image-based AI. Maybe get it to play chase with my daughters cats.

Connect a Raspberry Pi 4

To connect a Raspberry Pi, first you need to install Ubuntu, then connect it over USB-C, then configure the robot. All the instructions are in the Create3 compute board documentation, but I’ve included a basic run through here with some additional observations of things that caught me out!

Install and configure Ubuntu

ROS runs on Ubuntu, not on the default Raspberry Pi OS, so you need to start with a clean install of Ubuntu server on an SD card. From the Raspberry Pi imager you can create an image of Ubuntu Server 20.04. As I am using a Raspberry Pi 4, I used the 64-bit version. You can find this under Other general purpose OS -> Ubuntu

Once the image is created, you need to update some configuration on the image before you insert it into your Pi. The documentation tells you the changes to make, but the important gotcha is that the network-config file that you update to set your WiFi details will get copied on first boot into a yaml file, so you must get the indentation correct:

network:
 ethernets:
 eth0:
 dhcp4: true
 optional: true
 usb0:
 addresses:
 - 192.168.186.3/24
 dhcp4: false
 optional: true
 version: 2
 wifis:
 wlan0:
 access-points:
 "SSID":
 password: "password"
 dhcp4: true
 optional: true

I found the first time the Pi boots it won’t connect, it needs to boot a second time. Not sure why or if you will get the same.

It’s worth having a monitor and keyboard at the ready in case the Pi doesn’t automatically connect to your network. If it doesn’t, you will need to update the network configuration. To do this from the Pi, edit the /etc/netplan/50-cloud-init.yaml file, not the network-config in the root of the SD card - this file is only read the first time. Once you have updated the /etc/netplan/50-cloud-init.yaml file, reload the network plan using this command:

sudo netplan apply

If you configured everything correctly you should be able to connect to the internet. You can test this out by pinging a website.

ubuntu@ubuntu:~$ ping google.com
PING google.com (142.251.211.238) 56(84) bytes of data.
64 bytes from sea30s13-in-f14.1e100.net (142.251.211.238): icmp_seq=1 ttl=117 time=9.30 ms
64 bytes from sea30s13-in-f14.1e100.net (142.251.211.238): icmp_seq=2 ttl=117 time=7.76 ms
64 bytes from sea30s13-in-f14.1e100.net (142.251.211.238): icmp_seq=3 ttl=117 time=8.48 ms

Once you have network connectivity, continue installing all the required software listed in the documentation.

Configure the ROS middleware (RMW)

One of the last steps is to configure the ROS middleware, known as the RMW. This middleware is responsible for networking, and transmitting messages around. There are 2 options that the Create supports - CycloneDDS and FastDDS. I’d recommend FastDDS as it works with multiple networks out of the box. Your Pi will be configured with 2 network connections - a WiFi connection so you can program it, and a USB connection so it can talk to the Create3 robot. Out the box FastDDS can work with both of these, CycloneDDS does not unless you configure it.

Run this command to configure the Pi to use FastDDS.

echo "export RMW_IMPLEMENTATION=rmw_fastrtps_cpp" >> ~/.bashrc

Once the Pi is configured, you will need to configure the RMW on the Create3. To do this:

Set the robot into WiFi access point mode by pressing and holding buttons 1 and 2 at the same time till the LED flashed blue and the robot plays its happy noise.
Connect your computer to the WiFi from the Create3. It will be called Create-xxxx where xxxx is a mix of numbers and letters. The robot will play its happy noise when you connect.
Navigate to 192.168.10.1
Select Connect and set the WiFi so the Robot can connect to the internet. If you don’t do this, you will need to install an NTP server on your Pi to ensure the robot has the correct time. Messages are time based, so the robot and the Pi needs to have the same time. The network time documentation has the instructions for setting this up if needed. Note that running sudo chronyc clients to see the connected clients will return nothing if the Pi is not plugged in to the robot, and may take a while to show a connection the first time.
Select Application -> Configuration
Set the value of RMW_IMPLEMENTATION to rwm_fastrtps_cpp
Select Save then Restart Application. The robot will reboot.

Connect and test the Pi

Once the robot reboots, you can connect the Pi using a USB-C cable.

Open the top of the robot and ensure the USB/Bluetooth switch is set to USB. Then close the top.
Open the cargo bay to access the USB-C socket, and insert a USB-C cable.
Connect the other end of the cable to the USB-C socket on the Pi. This socket provides both power and connectivity.
If you want the Pi to be screwed in place, you can 3D print a caddy that the Pi connects to.
Close the cargo bay and wait for the Pi to boot up.

The default hostname for the Pi is ubuntu, but I’ve not been able to connect from macOS by sshing into ubuntu.local, so you will need the IP address. Either using your routers software to find it, or connect the Pi to a monitor/keyboard, boot it up and run ip a to get the IP address of the wlan0 network.

Connect to the Pi over SSH using the username ubuntu and password ubuntu. You will need to change the password the first time you log in.

Once you are in, you can verify the Pi is talking to the robot successfully buy running the following command:

ros2 topic list

This command lists all the topics that the Pi is able to see from the robot. These topics are how you can receive sensor data by subscribing to messages, and send commands to actuators.

The output should be:

/battery_state
/cmd_audio
/cmd_lightring
/cmd_vel
/dock
/hazard_detection
/imu
/interface_buttons
/ir_intensity
/ir_opcode
/kidnap_status
/mobility_monitor/transition_event
/mouse
/odom
/parameter_events
/robot_state/transition_event
/rosout
/slip_status
/static_transform/transition_event
/stop_status
/tf
/tf_static
/wheel_status
/wheel_ticks
/wheel_vels

The Pi can only see the topics if it was connected and booted when the robot boots up. If most of the topics are missing and you only see this list of topics:

/parameter_events
/rosout

Then you need to reboot your robot. The easiest way is to press buttons 1 and 2 to start the WiFi access point, then connect to it and select Application -> Reboot robot.

Your Pi is now connected and ready to program your first ROS application!

Led Ticker Tape

Fri, 25 Feb 2022 17:01:05 +0000

Anyone who knows me knows I’m a big fan of IoT and LEDs. I love using IoT devices to control lights, from the LEDs behind my desk to smart pumpkins!

I’ve been giving the .NET IoT libraries a spin for an upcoming project using a Raspberry Pi Zero W 2. I’m usually a Python person when using a Pi, but the project I’m working on needs a service that doesn’t have Python libraries that run on a Pi. Instead it has a .NET library, so it was time to break out my C# skills for the first time in years.

I wanted to build an LED panel that can display text, either static text or scrolling text. So I picked up this WS2812B panel from Amazon (affiliate link), and started to dig into the .NET IoT libraries. WS2818b LEDs are also known as NeoPixels, and are addressable multicolor LEDs, so you can light up individual ones in any color you like. They are addressed based on the order they are connected to your device, so the first pixel in a string of LEDs is 0, the next is 1 and so on. You can add as many as you like, and the addresses just keep going up.

The .NET IoT libraries are on GitHub at github.com/dotnet/iot and available as a NuGet. They support a wide range of boards including the Raspberry Pi.

Lighting LEDs in C#

I started as I usually do with a clean install of Raspberry Pi OS Lite. I use the Lite version as I access my Pi remotely using VS Code for all my development. You can read more on how I do this on my blog post on the Raspberry Pi blog. When I connected to my Pi I installed .NET 6, and the C# extension in VS Code.

One thing to note for the C# extension is it doesn’t support remote debugging on the Pi Zero W 2.

The .NET IoT libraries have a small amount of documentation and samples, so it wasn’t too much effort to get the LEDs lighting up.

The .NET libraries control these pixels over SPI, so they need to be connected to an SPI pin and ground on the Pi, as well as to a 5V power supply. You can’t use the 5V pins on the Pi as the panel can draw too much power and burn your Pi out, it’s best to use an external 5V power supply, either from a USB connection or a power converter plugged into the mains.

You also need to do a bit of SPI configuration, and this is documented in the GitHub source code for the WS2812B device code.

This is different to using the Adafruit NeoPixel libraries from Python, where you use different pins and don’t need any SPI configuration.

To use the .NET libraries, you start by creating an SPI configuration, then use that to create the pixels:

// Create connection settings to connect to the panel using SPI
SpiConnectionSettings settings = new(0, 0)
{
 ClockFrequency = 2_400_000,
 Mode = SpiMode.Mode0,
 DataBitLength = 8
};

// Create an SPI device
var spi = SpiDevice.Create(settings);

// Use the SPI device to connect to the LEDs
// and pass the number of LEDs
var neoPixels = new Ws2812b(spi, 256);

When you create the pixels, you pass in the length of the strip. I’ve been using a 8x32 panel, which is actually a 256 pixel long strip arranged in and up/down pattern.

Once created, you light pixels by getting a BitmapImage from them, and setting colors on that. This image is a length x 1 image, so 1 pixel tall, and as wide as the length of the LEDs. For example, my 8x32 panel is 256 pixels long, so is a bitmap of 256x1.

var img = neoPixels.Image;

You can then set pixels in this image to the color you want. The color is set using the System.Drawing.Color struct, which wraps ARGB values. The A (alpha channel) is ignored, so you set pixel brightness by reducing the value of the R, G, and B.

img.SetPixel(0, 0, Color.Red);
img.SetPixel(1, 0, Color.Green);
img.SetPixel(2, 0, Color.Blue);

Once the pixels are set in the image, it is committed and the LEDs updated.

neoPixels.Update();

One quirk of the bitmaps, is you have to set all the pixels up to the last one you want set. So if you want to set pixel 10 to be red, you have to also set pixels 0-9 to something, even if it is Color.Black (off). The first pixel you set is mapped to the first LED in the strip, so if you just set pixels 10-20, then the strip is set as if the first LED was pixel 10.

For example, if you just want to set pixel 2 to blue, you can’t do this:

var img = neoPixels.Image;

img.SetPixel(2, 0, Color.Blue);

neoPixels.Update();

What will happen here is pixel 2 is the first one with a value, so that will be considered the first pixel in the strip, so the first LED will light up blue. What you need to do is:

var img = neoPixels.Image;

img.SetPixel(0, 0, Color.Black);
img.SetPixel(1, 0, Color.Black);
img.SetPixel(2, 0, Color.Blue);

neoPixels.Update();

This will set the pixels 0 and 1 to off, and pixel 2 to blue.

Writing text

I wanted to make my panel show text, either short static text, or scrolling text. The first thing I needed was a font - something that defines how to create letters from pixels. I found a similar project based on Arduino in a GitHub project from Josh Levine so leveraged this code for a font and re-wrote it in C#.

Next I needed the mapping code. These fonts are defined as columns of binary data, so the bits to set. Each character is 8 bits tall (the size of my panel), and 6 bits wide. This mapping code was a bit of fun as I not only needed to divide up my pixels into columns, and map from a pixel in the 1 dimensional strip to a character pixel, but the strips go up and down!

The way this panel is made is by weaving an LED strip up and down, so the pixels start at 0 on the top left, go down to 7 on the left-most column, then across one pixel to the right to 8, then up to 15.

This gives for the first few columns:

This means that the mapping code needs to alternate - for 0dd numbered columns the pixels go down, for even numbered the pixels go up so the mapping has to be reversed!

I’m not going to dive into all this mapping here, but you can find the code with full documentation in my NeoPixelTickerTape GitHub repo.

I then added scrolling code that writes text starting at the right-most column, then re-writes it shifting left a column at a time.

Check out the code

You can find the code with full documentation in my NeoPixelTickerTape GitHub repo.

You can also download a NuGet package to use in your apps:

Auto-posting to dev.to using a GitHub action

Thu, 03 Feb 2022 00:00:00 +0000

I’ve been wanting to build a tool to post markdown automatically to blogging platforms. That way I (or anyone else) can write a blog post in markdown, save it in a GitHub repo, and have it automatically posted to a blogging platform of their choice.

I’ve created a small Python app to do this, and you can find it on GitHub at github.com/jimbobbennett/auto-blog-poster. You add some special folders to any folders containing README.md files, and it will create a blog post from the markdown. It will also track when the README file changes and update the blog post.

It’s slightly annoying to have to remember to run this every time you create or update a post, so I wanted to make it so it could be run automatically. GitHub actions are the perfect way to do this.

What are GitHub actions?

GitHub actions is GitHubs CI/CD solution. CI is continuous integration, meaning every time code changes, your code can be built and tested. CD is continuous deployment, so once code is tested it can be deployed automatically.

Essentially you can specify code that is run whenever someone checks in any changes, merges a branch or PR, or raises issues, creates PRs, any task really that you can do in GitHub. GitHub manages spinning up a VM to run everything, all you have to do is write your action, and pay (obviously - the best things in life are not always free).

GitHub actions are defined using YAML inside your repository (in a .github\workflows folder), and you can call out to actions that do things, such as checking out code, tagging, running scripts, anything you need. You can also build custom actions.

Custom actions

A custom action is one you write yourself to do whatever you need. In my case, I want my posting code to be run every time I update a markdown file in another repo, and this is something I can do with a custom action.

Custom actions are either written in JavaScript/TypeScript, or run from a Docker container. My app is Python, so I need to use Docker.

Creating a Docker custom action

Docker custom actions are Docker containers that can be run, and will stop when they are complete - you package up your app in a container, and provide it with an ENTRYPOINT so that Docker can run something.

I created a Dockerfile for my auto post tool, along with a shell script as the entrypoint:

# Create this docker file based off a Python 3.9 Linux image
FROM python:3.9-slim-buster

# Run everything from /app
WORKDIR /app

# Copy over the files
COPY requirements.txt requirements.txt
COPY app.py app.py
COPY dev_to.py dev_to.py
COPY github_access.py github_access.py
COPY entrypoint.sh entrypoint.sh

# Install the Python requirements
RUN pip3 install -r requirements.txt

# Execute the shell script as the entrypoint
ENTRYPOINT /app/entrypoint.sh ${@}

This Dockerfile creates a container using a Python base image, copies all my files over, installs my Pip package dependencies, then sets the entrypoint.

The interesting thing to note here is the parameter passed to the entrypoint.sh script - ${0}. My app needs some secrets passed to it - an API key for Dev.to to allow it to post, the repo to post from, and a GitHub token to allow it to update the repo once the post is up. I don’t want these embedded in the container as I want to be able to use this action from different repositories (and allow others to use it), so I want these passed when the action is run. The ${0} syntax means everything that is set as an environment variable when running the container, so this passes all the environment variables to the shell script, where they can be used in the app.

This means running the container like this:

docker run ghcr.io/jimbobbennett/auto-blog-poster:main
 --env DEV_TO_API_KEY=xxx
 GITHUB_ACCESS_TOKEN=xxx
 REPO=xxx/yyy

Will pass DEV_TO_API_KEY=xxx GITHUB_ACCESS_TOKEN=xxx REPO=xxx/yyy to the shell script, and this can be set as a local environment variable in the container.

Once created, I can build this container and publish it to the GitHub container registry from an action inside the repo for my post tool.

This is my action to publish my Docker container:

name: Docker Image CI

on:
 push:
 branches: [ main ]
 pull_request:
 branches: [ main ]

env:
 REGISTRY: ghcr.io
 IMAGE_NAME: ${{ github.repository }}

jobs:

 build:

 runs-on: ubuntu-latest

 steps:
 - name: Log in to the Container registry
 uses: docker/login-action@f054a8b539a109f9f41c372932f1ae047eff08c9
 with:
 registry: ${{ env.REGISTRY }}
 username: ${{ github.actor }}
 password: ${{ secrets.GITHUB_TOKEN }}

 - uses: actions/checkout@v2

 - name: Extract metadata (tags, labels) for Docker
 id: meta
 uses: docker/metadata-action@98669ae865ea3cffbcbaa878cf57c20bbf1c6c38
 with:
 images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}

 - name: Build and push Docker image
 uses: docker/build-push-action@ad44023a93711e3deb337508980b4b5e9bcdc5dc
 with:
 context: .
 push: true
 tags: ${{ steps.meta.outputs.tags }}
 labels: ${{ steps.meta.outputs.labels }}

This action is run on a push or PR on the main branch. It runs on an Ubuntu VM, logs into the GitHub container registry, checks out my code, gets the tag for the package, builds it and pushed it to the registry.

This code has some variables that come from GitHub. Any variable that starts ${{ github.xxx }} is set automatically by GitHub to a relevant value such as the repo name. ${{ steps.meta.outputs.xxx }} are set as outputs of certain steps, and ${{ secrets.xxx }} are secrets you can set on your repo. ${{ secrets.GITHUB_TOKEN }} is a special secret you don’t need to set that provides an API token to interact with the current repo.

Once this container is pushed, I can use it from an action inside my blog repo!

Using a Docker custom action

To use a Docker custom action, I can just pull it from inside my blog repo action and run it. Here’s the action YAML:

on: [push]

env:
 REGISTRY: ghcr.io

jobs:

 build:
 runs-on: ubuntu-latest

 steps:
 - name: Log in to the Container registry
 uses: docker/login-action@f054a8b539a109f9f41c372932f1ae047eff08c9
 with:
 registry: ${{ env.REGISTRY }}
 username: ${{ github.actor }}
 password: ${{ secrets.GITHUB_TOKEN }}

 - name: Use Docker CLI
 uses: actions-hub/docker/cli@master
 env:
 SKIP_LOGIN: true

 - run: docker pull ghcr.io/jimbobbennett/auto-blog-poster:main

 - run: >
 docker run ghcr.io/jimbobbennett/auto-blog-poster:main
 --env DEV_TO_API_KEY=${{ secrets.DEV_TO_API_KEY }}
 GITHUB_ACCESS_TOKEN=${{ secrets.GITHUB_TOKEN }}
 REPO=${{ github.repository }}

This action uses a docker custom action to log in to the GitHub container registry, pull my container, then run it, passing in an API key for Dev.to, the GitHub token and the current repository.

That’s it - now every time the blog post markdown changes in my repo, it is automatically deployed to Dev.to. At the moment this is just a playground, but the plan is to build out a new blog that uses this - all the posts will be in GitHub, and it will post to another blogging platform and Dev.to at every checkin.

Learn more

GitHub actions are a lot of fun, and a great way to set up CI/CD. I did a live stream where I worked all this out here:

You can also read more on the great GitHub Actions docs.

Getting started with GitHub Codespaces

Thu, 27 Jan 2022 00:00:00 +0000

The bane of every new developers life is getting your environment set up to be productive. And when I say new developer - I don’t just mean a dev who is new to a team, but every developer who needs to work on a project they haven’t worked on before.

Each project has it’s own dependencies, required tools, required libraries, a whole swathe of things that need to be installed, and can in some cases cause problems when projects have conflicting requirements. I certainly remember having to uninstall/reinstall different tooling versions when switching projects, sometimes multiple times a day 😱.

What if there was a way to fix this? If we could instantly have a pre-configured developer machine with the right tools that we need available at the click of a button? Even better one powered by the cloud so we don’t even need to worry about the power of our local machine, or even be able to access from a tablet or phone? This is where Codespaces comes in.

GitHub Codespaces are virtual developer machines in the cloud that you access through VS Code, running either on your desktop or in a browser. You can launch any GitHub repo inside a Codespace, with everything you do running in that Codespace - your code lives there, your debug sessions run there, your terminal runs commands there, it’s as if someone teleported a dev machine into your office!

Setting up a Codespace

I’ve been recently working on a project for my team that consists of a Python app that I want to run as a Docker container, so I thought it would be fun to configure the repo for this to run inside a Codespace so when anyone else on the team wants to work on it, they won’t have any local configuration to do.

Codespaces needs to be set up for a team or organization - mainly so someone can pay! Despite the claims that the best things in life are free, you do need to pay for Codespace.

Open your repo in a Codespace

The first step is to open the repo in Codespaces. From the repo in GitHub, select Codespaces from the Code button, then select New codespace.

This will set up your code in a new Codespace - essentially a blank VM using a default image from GitHub. This image is based off Ubuntu, and comes pre-configured with Python. Node, Docker and other stuff. You can read up on this default image at aka.ms/ghcs-default-image.

This image contains almost everything I need - the tools are all installed, and VS Code is running with my code in it.

My code is in Python, and this image comes with Python 3.8 installed. It means I can create a Codespace, and run my code in only a few seconds!

Configure your Codespace

The good news is Codespaces are configurable - you can define the details for the container in which your Codespace runs in a devcontainer.json file. I can use this to change the image used, configure what tools are installed, that sort of thing. The big upside of this is to ensure I have the right versions - the default container currently has Python 3.8 installed, but I can create a devcontainer file to set another version.

I’ll start by creating a devcontainer file. The Codespaces extension is installed for you in VS Code, so you can use the command palette to access options to configure the devcontainer file.

I started by selecting Codespaces: Add Development Container Configuration Files….

From there I selected From a predefined container configuration definition… to use a pre-defined image. I could also use any container I have in my container registry.

From the images I chose a Python 3 image, and selected Python 3.10.

I then had an option to add a Node version, so selected None as I don’t want Node.

Next I could select features to pre-install. I selected Docker as I need support for that.

2 things now happen. 2 new files are created in my explorer, devcontainer.json and Dockerfile in a folder called .devcontainer, and a toast will popup suggesting I rebuild the container. When I do this, Codespaces will restart with a new image based off my selections. It takes a while the first time as the container needs to be built.

The devcontainer.json file directs the Codespace to use the Dockerfile that was created to define the image. It then includes things like a list of extensions that VS Code will need - in my case PyLance.

"extensions": [
 "ms-python.python",
 "ms-python.vscode-pylance"
],

I could and any more extensions here if I wanted, such as the Docker extension.

I could also edit these files to create a virtual environment, install Python packages, that sort of thing - though I’m not sure I’d need a virtual environment as my container will only be used to develop from this repo, so I could install packages globally and not worry.

Check in your files

Once you are happy with your dev container setup, you can then check the .devcontainer folder and all it’s contents into your repo. This will then be used by anyone who creates a Codespace for your repo!

Learn more

If you want to learn more, check out the Codespaces docs - docs.github.com/codespaces. There was also some great videos on it from GitHub Universe, this particular one by Allison Weins and Bailey Brooks works through the configuration of Codespaces.

New year - new me!

Fri, 14 Jan 2022 22:44:32 +0000

🧀 Ah yes, a post title that’s cheesier than the best Parisian Fromageries.

Actually it’s not a new me, I’m still the same old me, but I have changed jobs! I’ve just started as a Regional Cloud Advocate working for the Microsoft Reactor team, based in Redmond, WA, USA.

A red neon sign saying Reactor Fun Zone

What is a ‘Microsoft Reactor’?

The Microsoft Reactors are our community spaces. We have 12 Reactors around the world, and we use these spaces to run events and bring communities together. We see these as spaces to learn and grow, to build your networks and build your skills.

The video below was filmed at the launch party for the Reactor London. The most important line in that video comes from Jeff Sandquist, the head of Developer Relations at Microsoft who says “I really hope that you don’t look at this space as a Microsoft facility, that you actually look at it as yours”.

But there’s no masks in that video?

Don’t worry, we’re all safe. That video was recorded pre-COVID, and to ensure the safety of our communities our Reactor spaces have pivoted to on-line. You may have watched some of our events already streamed on our YouTube or Twitch channels, or on LearnTV. Currently our locations are closed, but we are very much on-line, streaming a huge range of content from both Microsoft folks and community members.

Reactor landing page for a live streamed Global AI bootcamp in Toronto

So what’s the new job about?

My new job is to support communities, both on-line and in the local area. Redmond in Washington State, USA is the home of Microsoft, but in the surrounding Seattle area and beyond there are a huge array of tech communities thanks to the draw of tech companies both large and small, from Amazon to Zulily.

Cartoon racoons holding a laptop saying Community

I’ll be doing this in a number of ways. I’ll be running on-line streams to grow our Reactor meetup channel, with me learning or building with various technologies, I’ll be hosting community members and supporting their events, and I’ll be helping to grow out our Reactor as a space for communities to come together. I believe the the move from in-person to virtual, and the current thoughts on hybrid has given us the perfect opportunity to reset the traditional Meetup model, and build truly inclusive experiences for everyone - not just those who can make it to a venue at 6pm to drink beer, eat pizza and talk tech. I’ve had thoughts on this in the past, but really feel now we can all build more inclusive communities.

Reactor Redmond

The Reactor logo on the door of the Reactor in Redmond

The Reactor in Redmond, in Building 20 on the Microsoft campus is my home base. It’s a great space for events both small and medium sized, with a range of options for space to use. As we get more of a plan on re-opening I’ll post more with details on how you can sign up to use the space.

How can I use the Reactors?

Right now the Reactors are closed for events, but what we can offer is world class streaming facilities! We’ve been building out our AV (audio visual) tech to support streaming, so although we can’t bring your communities into our space, we can broadcast you from there to your communties. Get in touch if you live near a Reactor and want to discuss how we can support your streaming.

A panoramic view of the Reactor showing desks and Surface Hubs

We’d also love to host you on one of our streams. If you want to stream through the Reactor meetup group, again get in touch and we’ll get you schedued. We have Reactors around the world, so we can fit into your time zone.

What about IoT?

Don’t worry, I still love IoT and I’ll be streaming a load of IoT content. This new role won’t change my technical focus, it just means I can work with a wider audience.

Join us!

Come join our meetup group - you can sign up on Meetup.com:

https://www.meetup.com/pro/microsoft-reactor

Announcing a New Free Curriculum: IoT for Beginners

Fri, 09 Jul 2021 00:00:00 +0000

It is our very great pleasure to announce the release of a new, free, MIT-licensed open-source curriculum all about the Internet of Things: IoT for Beginners. Brought to you by a team of Azure Cloud Advocates, Program Managers, and Microsoft Learn Student Ambassadors, we hope to empower students of all ages to learn the basics of IoT. Presuming no knowledge of IoT, we offer a free 12-week, 24-lesson curriculum to help you dive into this amazing field.

If you liked our first two curricula, Web Dev for Beginners and Machine Learning for beginners, you will love IoT for Beginners!

Join us on the journey of your food, from farm to table!

🌽 Join us as we take the same journey as your food as it travels from farm to table, taking advantage of IoT on the way to improve farming, transport, manufacturing and food processing, retail and smart homes. 🌽

Our curricula are structured with a modified Project-Based pedagogy and include:

a pre-lesson warmup quiz
a written lesson
video
knowledge checks
a project to build
infographics, sketchnotes, and visuals
a challenge
an assignment
a post-lesson quiz
opportunities to deepen your knowledge on Microsoft Learn

What will you learn?

The lessons are grouped so that you can deep-dive into use cases of IoT. We start with an introduction to IoT, covering devices, sensors, actuators and cloud connectivity, where you will build an internet connected version of the “Hello world” or IoT, an LED. We then move on to farming, learning about digital agriculture and feedback loops to control automated watering systems. Your food then leaves the farm on trucks, and you learn how to track vehicles using GPS, visualize their journeys and get alerts when a truck approaches a processing plant. Once in the plant, we move to AIoT, learning how to distinguish between ripe and unripe fruit using AI models running from IoT devices and on the edge. Next we move to the supermarket, using IoT to manage stock levels. Finally we take the food home to cook, and learn about consumer smart devices, building a voice controlled smart timer that can even speak multiple languages.

Hardware

The hard part (pun intended) for IoT is hardware, so we’ve designed this curriculum to be as accessible as possible. We want you to Learn IoT, not learn how to solder, know how to read resistor color codes, or know what a microfarad is, so we’ve made hardware choices to make things easier.

You can choose to learn using microcontrollers using Arduino with a Wio Terminal, or single board computers using a Raspberry Pi. We’ve also added a virtual hardware option so you can learn without having to purchase anything!

For sensors and actuators, we’ve gone with the Grove kit from Seeed Studio, with easy to connect sensors and actuators.

Our friends at Seeed have made it easy to buy the hardware, with packages containing all of the kit you need.

If you are interested in learning using virtual hardware, you can write IoT code locally as if you were using a Raspberry Pi, then simulate sensors and actuators using CounterFit, a free, open source tool for simulating hardware.

A sneak peek

This curriculum is filled with a lot of art, created by our team. Take a look at this cool sketchnote created by @nitya.

Without further ado, please meet IoT For Beginners: A Curriculum!

Using TinyML to identify farts

Mon, 22 Feb 2021 17:01:05 +0000

TLDR; Find a complete hands-on lab to build a TinyML audio classifier at github.com/microsoft/iot-curriculum/tree/main/labs/tiny-ml/audio-classifier.

My 8-year-old daughter bought me “Farts - a spotters guide” - a book with some buttons down the side and when you press them, they make different fart sounds. This is the height of humor for an 8 year old, and still pretty funny as an adult. I thought it would be fun to see if I could distinguish between the different fart noises using machine learning - and not just any machine learning, but seeing as I love IoT, I wanted it to run on a microcontroller!

TinyML

TinyML is a relatively new field, and is all about creating tiny machine learning models that can run on microcontrollers. These models are really tiny - in the order of kilobytes instead of the usual megabytes or gigabytes. They need to be this tiny to run on microcontrollers that typically have kilobytes of RAM. These models also draw little power, typically in the single-digit milliwatts or lower.

What are the use cases for TinyML? Well there are loads, anywhere you want to run ML models offline with minimal power draw. You may even have some TinyML models running in your house right now. For example, smart voice controlled devices listen for a wake word, and this needs to be offline and draw minimal power - perfect for a TinyML model. Another use case is in healthcare with devices that can monitor your health that run for years on tiny batteries. It’s also being used in animal smart collars and trackers, using audio to monitor the health of elephants in the wild. So yes - a fart detector has a real world application!

To build a TinyML model you need to decide what type of model to build, gather training data, train the model, then deploy it to your device to handle new data. In this case, I wanted an audio classifier, so decided to use a support vector machine classifier.

Despite this sounding all fancy and like I know what I’m talking about, I actually have no clue what this is - I just learned about them from a great tutorial which I followed to get inspiration for this post! The tutorial is Better word classification with Arduino Nano 33 BLE Sense and Machine Learning.

Building a fart detector

For my fart detector, I needed to build an audio classifier that could run on a microcontroller. Because I’m terrible at electronics and understanding I2C, SPI and all that other stuff, I decided to use an all-in-one Arduino board that has a microphone built in allowing me to use off-the-shelf Arduino libraries to gather audio data. The board of choice was the Arduino Nano 33 Sense BLE board, a small Arduino board with a whole raft of sensors including a microphone, temperature, pressure, humidity, light level and color, gesture and proximity. That’s a lot of sensors in such a tiny board!

To code this board, I could use the free Arduino IDE, but I prefer to use Visual Studio Code, along with the PlatformIO extension. This allows the creation of standalone microcontroller projects with .ini files that define the board and libraries used. I can check a project into GitHub and someone can clone it and immediately start working with it without the need for instructions on what boards and libraries they need to set up.

Getting training data

To train TinyML models you not only need the model to by tiny, but you also need small inputs - the more data that goes into training the model or inference (that is running the model), the larger it is. Audio data can be quite large - for example CD quality audio (remember CDs?) is 44.1KHz/16-bit which means it captures 2 bytes of data 44,100 times per second, or 176KB per second! That’s a lot of data - if we wanted to use all of it and train our model with 2 seconds worth of data it wouldn’t be TinyML any more.

A great trick with audio data is realizing you don’t need all of it to classify particular sounds. Instead you can get an average value that represents many samples and use that as the data. In the case of the Arduino, the library that captures audio, PDM, captures audio at 16KHz in 512 byte buffers, containing 256 2-byte samples. This means each buffer has 1/64th of a second of audio data in it. We can then calculate a root mean square (RMS) of all this data to get a single 4-byte floating point value. If we do this for every buffer, we end up with 64 4-byte floats per second, or 256 bytes per second. Much smaller than raw audio at the PDM sample rate of 16KHz giving 32,000 bytes per second!

#define BUFFER_SIZE 512U
...
// Check we have a full buffers worth
if (PDM.available() == BUFFER_SIZE)
{
 // Read from the buffer
 PDM.read(_buffer, BUFFER_SIZE);

 // Calculate the root mean square value of the buffer
 int16_t rms;
 arm_rms_q15((q15_t *)_buffer, BUFFER_SIZE/sizeof(int16_t), (q15_t *)&rms);
 ...
}

The RMS value can be checked against a threshold to see if there is actual audio data or not, and if audio data is detected, the next 2 seconds worth can be grabbed. In this case it’s output to the serial port so it can be read from the PlatformIO serial monitor in VS Code.

You can find the full code to capture audio samples in the Microsoft IoT Curriculum resource GitHub repo in the labs folder.

Train the model

To train the model, we need a good range of audio data captured from the Arduino device - ideally 15-30 samples per audio we want to classify. A classifier distinguishes the input between multiple labels, so we need to gather data for multiple labels. For example, to classify the farts from my fart book I’d need to gather 15-30 samples for at least 2 different farts.

The audio data sent to the serial monitor from the Arduino can be captured into .csv files, and these can be loaded by a Python script and used to train a model.

The model in question is trained using Scikit-Learn, a Python Machine Learning library. The audio data is loaded into numpy arrays, then split into training and testing data, the model is trained using the training data, then tested with the testing data to give an idea on the accuracy.

If you have a nice shiny Apple M1 mac (like I do), then installing Scikit-Learn is currently not as easy. Check out my guide on how to install it

# Split the data into a training and testing set to test the accuracy of the model
# If you are happy with the accuracy of the model, you can remove this split
dataset_train, dataset_test, label_train, label_test = train_test_split(dataset, dataset_labels.ravel(), test_size=0.2)

# Build the support vector classification for our data and train the model
svc = SVC(kernel='poly', degree=2, gamma=0.1, C=100)
svc.fit(dataset_train, label_train)

# Test the accuracy of the model
print('Accuracy:', svc.score(dataset_test, label_test))

Once the model has been trained, it can be exported using the rather useful micromlgen Python library which can convert ML models into raw C++ code to run on microcontrollers.

from micromlgen import port
...
# Convert the model to C code and write to the classifier.h file
c_code = port(svc, classmap=label_map)
with open('classifier.h', 'w') as f:
 f.write(c_code)
 f.close()

You can find the training code in the Microsoft IoT Curriculum resource GitHub repo in the labs folder.

Classify farts

The C++ code that comes out of the training can then be added to the microcontroller code. Instead of dumping the audio data to the serial port, it can be sent to the classifier code, and the label of the best match is returned.

void processSamples()
{
 // Write out the classification to the serial port
 Serial.print("Label: ");
 Serial.println(clf.predictLabel(_samples));
}

Learn more

You can find a complete hands on lab implementing this in the Microsoft IoT Curriculum resource GitHub repo in the labs folder.

Installing Scikit-Learn on a Apple Silicon

Sun, 31 Jan 2021 17:01:05 +0000

At the end of last year I splashed out on a shiny new Apple MacBookAir with the M1 processor as I was fed up with an old Intel-based MacBookPro that was quite honestly crippled by corporate anti-virus software.

Out the box this machine is amazing. It’s ridiculously fast, and lasts for ever on battery. Seriously - I charge it every 2 days and manage a full day of coding, writing, emails, Teams, the lot. Did I also mention it’s fast? I can have all the things running and it barely breaks a sweat, even with only 8GB of RAM.

The downside is that not all software works on the new ARM-64 architecture. Apple have a translation layer called Rosetta 2 (Rosetta 1 was their translation from PowerPC to Intel), and this works great most of the time for every day apps, but it doesn’t always work for development tools and libraries, as the mix of translated and untranslated stuff just breaks down.

One library I needed to use that isn’t supported is Scikit-Learn. Now I’m no Python expert, and I don’t really understand what Scikit-Learn does, I just know I need it to train some TinyML models to recognize wake words on an Arduino Nano 33 sense board. If I try a normal pip install scikit-learn, I get a whole wall of errors, both using Python 3.9 for the M1, and Python 3.8 under Rosetta.

So what to do?

It turns out the solution is to use Miniforge, a version of Conda that is comparable to Miniconda, but supports various CPU architectures. Whatever that means. As I said, I’m no Python expert, but this tool essentially allows me to create virtual environments and install packages compiling them for the M1 chip! Any packages it doesn’t support can then be installed from pip.

So how do I install all this?

Firstly - I need to install Miniforge. The install script is on the GitHub page, or you can download it by clicking this link. It wanted to activate it in every terminal, which I didn’t want so I turned that off by running:

conda config --set auto_activate_base false

Next I went to the folder containing my Python code, and created a virtual environment:

conda create -n .venv python

This is pretty much the same as creating a virtual environment with Python, just using a different tool. Like with Python, the virtual environment then needs to be activated:

conda activate .venv

Finally, I can install Scikit-Learn:

conda install scikit-learn

Done! For the particular thing I’m working on, I needed a package that isn’t available from Miniforge, so I just installed it with pip:

pip install micromlgen Done! I could then run my Python script as normal, and it all worked nicely. And fast - my M1 ran the script in question in 2 seconds, 5 times faster than the 10 seconds my Surface Book took.

Build a virtual IoT Lab with Raspberry Pis and Azure IoT Hub

Fri, 11 Dec 2020 00:00:00 +0000

The rise of virtual education has led to sweeping changes in how students are taught. A lot of computer science lessons can be run with only small changes thanks to the cloud - Azure Lab Services can provide virtual machines with developer tooling, or GitHub Codespaces can replace the physical lab setups that Universities used to use.

One area that has limitations is the Internet of Things. You can’t move small microcontrollers or small board computers to the cloud, posting devices is expensive and leads to support issues that often can only be resolved by posting the device back, and asking students to purchase devices is problematic as this can be expensive for a group of students who may already be paying many thousands of dollars for tuition, and in some cases not feasible due to shipping or customs issues.

Although it is impossible to re-create an IoT lab fully in a virtual setting, it is possible to make devices like Raspberry Pi’s available over the internet, either from a University campus, or even from the Lecturers home (given enough upload bandwidth), and these can then be connected to microcontrollers to program these if needed. There are plenty of solutions to do this that involve opening SSH ports over the internet, but these are not the most secure as you have to have a public SSH port open.

One novel way that increases security is using Azure IoT Hub device streams.

Use device streams to proxy an SSH connection

Device streams provide secure two-way TCP tunnels via standard HTTPS connections - allowing devices to bypass firewalls and other network considerations, and avoid having SSH ports open over the internet, but still have security via IoT Hub.

You can then run server code on your local device to emulate SSH, and it can forward the connection to a device stream via IoT Hub, which streams to the client IoT device which is running client code to listen to the requests over the stream and redirect them to the local SSH port, and stream back the results, essentially using very thin proxies.

Although the Pi is connected to an IoT Hub to stream the SSH commands, it can still connect to another IoT Hub for IoT development, and run services like Azure IoT Edge. From a users perspective, it’s the same as SSHing into the Pi - they connect an SSH session to the local proxy using a different port, and that logs them into the Pi. Behind the scenes the IoT Hub device streams make it work, but to the user, it’s as if they connected natively.

The full process for how to set this up, as well as all the code you need to build and run the proxy is available as one of our Lab Guides as part of the Microsoft IoT Curriculum resources. Check out the instructions and try it out.

github.com/microsoft/iot-curriculum/blob/main/educator-guides/lab-guides/virtual-iot-lab/iot-hub-ssh-virtual-lab.md

What is edge computing, why do it, why send IoT data to the cloud?

Thu, 03 Dec 2020 00:00:00 +0000

I recently had a student reach out to me with some great questions around Edge computing and how it matches to IoT, and indeed why even use the cloud with IoT. They have to write a paper on the difference between edge computing and just using the cloud, and were researching these terms and trying to understand the whys and hows.

There is currently a lot of confusion around this topic, especially when product pages are full of buzzwords, marketing speak and business decision maker language, not student friendly explanations, so I thought I’d take a moment to try to answer their questions with a blog post and hopefully help others navigate this minefield.

This blog post is a mind dump in the style of I didn’t have time to give you a short answer, so I wrote a long blog post instead, so comments and criticism welcome.

Temperature detection

I’m going to frame all this with the canonical IoT example of temperature detection. It’s one of the ’easiest’ IoT scenarios in that there are a huge range of devices and examples out there for this. You take an IoT device such as a micro controller (for example an Arduino board) or a single-board computer (for example a Raspberry Pi), attach a temperature sensor, and gather the data.

What is cloud computing?

Lets start with the cloud. The cloud is someone elses computer - you pay per use for either computing resources or software resources managed by someone else. it allows you to not worry about purchasing hardware, managing cooling, electricity and networking, managing software, security, patching and the other day to day operations, and instead outsources these to experts. The cost comes down - you only pay for what you need when you need it, and speed of delivery goes up as the services are pretty much instant on.

In the IoT space for example, a few clicks brings you a service you can securely send IoT data to, and a well defined way to get the data back out for analysis, all for a modest monthly fee. Running the same setup manually would be highly expensive and a lot of work.

Why send this data to the cloud?

A great question - why use the cloud when you can just read the data yourself, either by showing it on a screen or even by connecting to a Raspberry Pi and reading the values?

The answer lies in the rise of the smart thermostat. In the house I grew up in, the temperature sensor was a dumb device - it detected the temperature and if it was lower than a value on a thermostat, the heating turned on. The next generation was connected thermostats - sending the data to the cloud. This added a level of usability to these devices - yes you could walk to the thermostat and check the temperature of your house, but you could also get this on your phone. Combine this with the ability to also control the thermostat remotely you have a great advantage of using the cloud. You can check the temperature of your house from anywhere and control it from anywhere. On your way home unexpectedly on a cold day? You can check how cold your house is and turn the heating up if it’s set to a colder vacant setting. The cloud starts to take your data and control anywhere.

Yes, if you don’t care and just want a wall thermometer then there’s no need, but as soon as you want access away from the temperature sensor, the cloud wins.

The next generation of smart thermostats is around analyzing this data in the cloud. A disconnected device relies on manual control, a cloud connected device can use algorithms to make control decisions for you based on a wide variety of data. Your cloud connected thermostat can check your calendar, and if it sees you are on vacation it can turn your heating off. It can also check weather, and if the heating is off but a cold snap is coming it can turn it on to stop pipes freezing. All this comes via having the data and control in the cloud.

This idea can be taken further with temperature monitoring of factories and machinery - sometimes subtle variations in temperature of a machine component can indicate an upcoming failure. By constantly monitoring the temperature, with data sent to AI models, you can be do predictive maintenance, replacing a part early before a costly failure.

The downside of the cloud

The cloud isn’t perfect, and can have a couple of downsides - though these can be mitigated.

Reliance on the cloud

Recently one of the big cloud providers had an outage, and social media was full of complaints from people who can’t vacuum their house as the cloud was down. If your device relies on the cloud, it also needs to work when the cloud is not there - I don’t want a WiFi outage or a data center fire to stop me from having heating, especially if I lived in a really cold location! There are also locations where internet connections are unavailable or expensive, such as oil rigs or deep underground, or even in space.
Data privacy

The cloud is secure, and the big clouds have a lot of certifications and approvals for data security, but there might be times when you don’t want data in the cloud. For example, some countries insist on personal data remaining in-country (data sovereignty laws). If the cloud doesn’t have a data center in your country, you have to store it locally or on a local data center. Some data, such as medical data might need to stay on-site in a hospital.
Bandwidth and network speed

Bandwidth isn’t free. If you want to run AI models to analyze video from security cameras, sending all that data to the cloud is going to need a lot of bandwidth. That’s expensive, and not always available. The internet is also not as fast as internal networks, so if you need the data or results of analysis quickly then it might be faster to have the analytics closer to the data capture.
Cost

The cloud brings the cost down, but sometimes you don’t want or need to pay for the power of the cloud. It might be that a $99 NVIDIA Jetson Nano can run your AI models fast enough that you don’t want to rent a $100 a month GPU-powered VM in the cloud.

This is where Edge computing comes in.

What is edge computing?

Edge computing is running parts of the cloud on the edge - that is on your own network in your building or data center. You can take advantage of the cloud to build and train workloads, then deploy them to an edge device to run closer to your data. This way you get the benefit of the scalable power of the cloud to train complex models, and the advantage of local compute to:

Avoid dependency on the internet and the cloud - your models can run offline, so if the cloud or internet goes down, or if you are away from the internet such as at sea, they keep running.
Keep your data private and local
Not be limited by external bandwidth and internet speed, only be limited by your internal network bandwidth
Control costs using existing hardware

Thinking about factory monitoring - you can do this so much better on the edge than in the cloud. You can gather the data you need and train complex AI models for predictive maintenance in the cloud. Once trained, you can download these models to an IoT Edge device built using cheap hardware. This device can be on your local network close to the machinery, and respond to temperature data - instantly alerting someone or even turning the machinery down or off it it detects a possible failure. If the internet or the cloud goes down - the device still works. If you send millions of data points to it for analysis, you are not limited by outgoing internet speeds.

These models running on the edge are not static - you can constantly improve and retrain models in the cloud and deploy them to the edge as required.

You can also use these edge devices as gateways - sending data up to the cloud as needed. This can be filtered data, such as removing duplicate values or data within an allowed range, or they can be used to route data from devices that can’t connect to the cloud services directly. They can even shield devices from being connected directly to the internet of these devices are not secure.

To visualize how this works - I’m going to defer to an article from the Microsoft Documentation covering running Stream Analytics on IoT Edge. Stream Analytics is a tool for creating real-time queries against streaming data, outputting the results of the query to other systems. Stream Analytics jobs can be run in the cloud, or downloaded onto an IoT Edge device and run on the edge.

This diagram shows IoT Edge devices running Azure Stream Analytics jobs via the IoT Edge runtime, running on-premise in a factory, with data coming in from a variety of devices. The results of the Stream Analytics job are then sent on to Azure IoT Hub for further analytics if needed.

More resource

If you want to learn more about this topic - here’s some great resource:

Azure IoT Edge product page
Azure IoT Edge documentation
Run cognitive services on IoT Edge - a video where I am joined by Marko Paloski and we talk about running a pre-built AI service on the Edge using Azure IoT Edge
AI Edge engineer learning path on Microsoft Learn - a learning path produced in partnership with the University of Oxford covering AI on the edge
Build the intelligent edge learning path on Microsoft Learn - a learning path covering how to use Azure IoT Edge to build IoT solutions that require having cloud intelligence deployed locally on IoT Edge devices
Microsoft IoT Curriculum resource - a GitHub repo of resources, curated links and labs for IoT classes, projects and learning
Assembly line QA lab - a hands-on-lab covering how to use IoT edge to do AI-powered assembly line validation

Control holiday lights with Python, Azure IoT and Power Apps

Fri, 13 Nov 2020 00:00:00 +0000

As the nights draw in here in the northern hemisphere, there are a number of winter celebrations that happen - and one thing they all have in common is lights.

In the past I’ve just purchased strings of lights from the nearest retailer, strung them up to a tree or around my house, and fought with remote controls or buttons to get the color I wanted.

Well this year I decided to do something better and build IoT powered holiday lights, controlled by a Power App built with no code!

The hardware would be based around WS2812B programmable LED strips (also referred to as Neopixels), controlled by a Raspberry Pi Zero W. Software wise, the Pi would run some Python code to talk to Azure IoT Central - an IoT software as a service platform that can send commands to the Pi to turn the lights on or off. A Power App would then be used to control IoT Central via a mobile app - written with no code! All these cloud services can be used for free, which is even better!

You can find a full hands-on guide to building this your self with detailed instructions and all the code in my Neopixel Holiday Lights GitHub Repo

Hardware needed

To create this, you will need the following hardware:

A Raspberry Pi that can connect to the internet such as a Pi Zero W or a Pi 3/4, with appropriate SD card and power supply
A strip of WS212B programmable LEDs
A 5v power supply - either a USB to terminal block lead or a 5v DC converter
Jumper wires

To wire the hardware, connect the positive lead or pin on the LEDs to the positive terminal on the 5v power supply, Connect the negative lead or pin to both the negative terminal on the power supply, and a GND GPIO pin on the Raspberry Pi. Some LED strips have twin negative leads for this reason, otherwise you’ll need to solder or attach more leads as necessary. Finally attach the control pin to GPIO pin 18 on the Raspberry Pi. You can find details on the pin numbers in the Raspberry Pi GPIO documentation.

Azure IoT Central

Azure IoT Central is a software as a services platform that allows you to build the cloud infrastructure and security for your IoT apps with no code. It is based around device templates - a description of the data that a device can send to the cloud, and commands the cloud can send to the device to instruct it to do something.

Start by heading to apps.azureiotcentral.com and creating a new app. Although IoT Central is a paid service, you can use it for free. There is a free tier, but apps created using this are deleted after 7 days, so go for one of the standard tiers. Both the standard tiers offer 2 devices for free, so you can control your Pi without paying anything.

You will need an Azure Subscription to sign up, so if you don’t have one you can sign up for free. Students can sign up at azure.microsoft.com/free/students to get $100 of credit for a year, renewing every year, otherwise sign up at azure.microsoft.com/free to get $200 of credit that lasts 30 days.

Once you have your app created, create a device template with 2 commands and a property. The commands will turn the lights on and off, and the property will allow the color to be persisted so restarting the Pi means the lights will re-light with the last color set.

Name one of the commands On and make it take a request parameter called Color. Name the other command Off with no request parameter. Create the property called Color as a string. Publish this template, then create a new device using it. Once you’ve created the device, grab the connection details.

Program the Pi

Next step is to program the Pi to connect to IoT Central, receive the commands and turn on the lights. This can be done in Python. Make sure your Pi is using the latest Raspberry Pi OS, with all the latest updates and is connected to the internet. You can use either the Lite or Desktop version - for a project like this where the Pi is running headless I like to use the Lite version to allow me to use smaller SD cards and boot the Pis faster.

To program the Pi, you can use your favorite Python tool. I personally recommend using Visual Studio Code, a free, open source developer text editor that has extensions for a variety of languages, including the PyLance extension for Python. If you are using the full desktop version of Raspberry Pi OS, you can install VS Code locally, otherwise use the Remote SSH extension to code remotely on a Pi 3 or 4, or the SSH File system extension to code remotely on a Pi Zero.

First you need to install some Pip packages. These need to be installed using sudo - to control the Neopixels you need to run the code as a super user. Install the following:

rpi_ws281x
adafruit-circuitpython-neopixel
azure-iot-device
python-dotenv

The first 2 packages provide control for the LEDs, the azure-iot-device package has the Python code for devices to connect to IoT Central, and python-dotenv is used to load secrets from .env files to avoid things like API keys being uploaded to source code control.

Once these are installed, create a file called .env on the Pi in whatever folder you want to create the code in to store the connection details for the IoT Central device, and add the following:

ID_SCOPE=<ID scope>
DEVICE_ID=<device id>
PRIMARY_KEY=<primary key>

Replace <ID scope> with the value of the ID Scope from the IoT Central device connection dialog. Replace <device id> with the device ID, and the <primary key> with the primary key.

Next create a file called app.py and add the following code. You can also find this code in the GitHub repo tha accompanies this post.

import asyncio
import board
import neopixel
import os
from azure.iot.device.aio import IoTHubDeviceClient, ProvisioningDeviceClient
from azure.iot.device import MethodResponse
from dotenv import load_dotenv

# Define the NeoPixel strip setting:
# The pin the control wire is connected to (18 in this code)
# The length of the strip (150 LEDs in this code)
# The brightness (0.2 on a scale of 0-1)
# If the colors are written as soon as the values are updated, or if they need to be
# updated all at once as soon as the values are set
pixels = neopixel.NeoPixel(board.D18, 150, brightness=0.2, auto_write=False)

# Load the IoT Central connection details from a .env file
load_dotenv()
id_scope = os.getenv('ID_SCOPE')
device_id = os.getenv('DEVICE_ID')
primary_key = os.getenv('PRIMARY_KEY')

# Declare the device client so it can be used from all the function
device_client = None

# Provisions the device with the Azure device provisioning service or returns
# the connection details if the device is already provisioned
async def register_device():
 provisioning_device_client = ProvisioningDeviceClient.create_from_symmetric_key(
 provisioning_host='global.azure-devices-provisioning.net',
 registration_id=device_id,
 id_scope=id_scope,
 symmetric_key=primary_key,
 )

 return await provisioning_device_client.register()

# Sets the color of the Neopixels based on a color string coming in.
# This color string is a 6 character code, 2 characters for red, 2 for green
# and 2 for blue. These 2 characters are a HEX value from 00 to FF.
# For example FF0000 is full red, no green or blue. FFFFFF is white, 000000 is off.
# Once the color is set, write it back to the IoT Central property via a device twin
async def set_color(color):
 # split in the color string into the red, green and blue components, and convert these
 # to valid hex strings
 r = '0x' + color[0:2]
 g = '0x' + color[2:4]
 b = '0x' + color[4:6]

 # Convert hext to numerical values
 r_value = int(r, 0)
 g_value = int(g, 0)
 b_value = int(b, 0)

 print('Updating color: r =', r_value, ', g =', g_value, ', b =', b_value)

 # Set all the pixels to the new color
 pixels.fill((r_value, g_value, b_value))

 # Show the color on all the pixels
 pixels.show()

 # Write the color back as a property
 # Properties are written to the device twin, so patch the reported properties
 # with the color
 patch = {'Color':color}
 print("Sending patch:", patch)
 await device_client.patch_twin_reported_properties(patch)

# IoT Central command handler
# IoT Central commands are implemented as IoT Hub direct methods
async def command_handler(method_request):
 print("Message received:", method_request.name)
 print("Message payload:", method_request.payload)

 # Determine how to respond to the command based on the IoT Hub direct method method name
 # which is the same as the IoT Central command name
 if method_request.name == "On":
 # For an On request, set the color based on the payload
 await set_color(method_request.payload)
 print("executed on")
 elif method_request.name == "Off":
 # For an Off request, set the color to 000000, which turns the pixels off
 await set_color("000000")
 print("executed off")
 else:
 print("Received unknown method: " + method_request.name)

 # Method calls have to return a response so IoT Central knows it was handled correctly,
 # So send a 200 response to show we handled this
 payload = {"result": True}
 status = 200

 # Send the response
 method_response = MethodResponse.create_from_method_request(method_request, status, payload)
 await device_client.send_method_response(method_response)

async def property_handler(patch):
 print("Patch received:", patch)
 if 'Color' in patch:
 await set_color(patch['Color'])

# The main async function that runs the app
async def main():
 global device_client

 # Regsiter the Pi as an IoT device in IoT Central
 registration_result = await register_device()

 # Build the IoT Hub connection string from the registration details
 # IoT Central sits on top of IoT Hub, and the Python SDK only supports IoT Hub,
 # So to talk to IoT central the IoT Hub connection string needs to be built from details
 # from registering the device with the provisioning service
 conn_str='HostName=' + registration_result.registration_state.assigned_hub + \
 ';DeviceId=' + device_id + \
 ';SharedAccessKey=' + primary_key

 # The client object is used to interact with your Azure IoT Central app via IoT Hub, so create this 
 # from the connection string
 device_client = IoTHubDeviceClient.create_from_connection_string(conn_str)

 # Connect the client to IoT Hub
 print('Connecting')
 await device_client.connect()
 print('Connected')

 # IoT Central stores properties in the device twin, so read this to see if we have a color
 # stored from the last run for the lights. This way when the device starts up it can set the color
 # to the last setting
 twin = await device_client.get_twin()
 print('Got twin: ', twin)

 # Load the color from the reported properties of the twin if it exists
 if 'reported' in twin and 'Color' in twin['reported']:
 await set_color(twin['reported']['Color'])

 # Set the method request handler on the client to handle IoT Central commands
 device_client.on_method_request_received = command_handler

 # Handle updates to the color property from IoT Central
 device_client.on_twin_desired_properties_patch_received = property_handler

 # Define a message loop that keeps the app alive whilst listening for commands
 async def main_loop():
 while True:
 await asyncio.sleep(1)

 # Wait for user to indicate they are done listening for method calls
 await main_loop()

 # Finally, disconnect
 await device_client.disconnect()

# Start the async app running
if __name__ == "__main__":
 asyncio.run(main())

Read the code and the comments to see what it does.

This code will connect to the Azure device provisioning service to authenticate the device using the settings from the .env file, then connect. It will load the properties to see if a color has already been set, and if so set the Neopixels to that color. It will then listen for commands to change the color of the Neopixels or turn them off.

Run the code as sudo with the following command:

sudo python3 app.py

The app will start up and connect. You can then run the commands in IoT Central to control the lights.

The color for the On command should be given as a string value representing a Hex string. It will be 6 characters long, with 2 characters each representing the R, G and B values with values from 0-255 (00 - FF). For example, red is FF0000, green is 00FF00, blue is 0000FF, yellow is FFFF00, white is FFFFFF.

Set up a Power App

Power Apps allow you to build apps with low or no code, by creating flows that connect logic and services, and connecting these to a UI designed on a canvas.

You will need a Power Apps account to create one, and if you don’t have one you can sign up for a community plan. This will give you a free environment for learning and building your own apps.

Start by creating a flow in the Power App. This should use the Create from Template option selecting Power Apps button. Name this flow Turn lights on and add an Azure IoT Central V3 connector (NOT the V2 connector). Select your IoT Central app, add the device id, set the device template, select the capability, then select the On command. A new box will appear for the color to pass to this command, so select this box and select Ask in Power Apps from the box that appears. Then save the flow.

Create another flow for the Off command called Turn lights off.

Once you have the flows, create an app, Drag a text input and two buttons to the canvas and align then in a column.

Name the text input ColorInput, remove the default value and change the hint to color.

Name one button On and select the Action tab on the ribbon, then select Power Automate. From the popup, select the Turn lights on flow. It will take a few seconds to add the flow. When it does a half complete function will appear in the function bar. Pass the Text property of the ColorInput control into:

PowerAppsbutton.Run(ColorInput.Text)

Name the other button Off and add the Turn lights off flow to it. The function for this doesn’t need any parameters, so just close the brackets.

Make sure the app is running on the Pi, then save and test the Power App using the Preview this app button on the toolbar.

Done!

Your app is now complete! Mount the lights somewhere and control them from the Power App. You can you can run it on your phone. Install the Power Apps app from the Apple App Store or Google Play Store, log in, select your app and use it to control your lights.

Learn more

You can learn more about Azure IoT Central and the Microsoft Power Platform via Microsoft Learn, a hands-on, self-guided learning platform from Microsoft.

Run Visual Studio Code on a Raspberry Pi

Sun, 25 Oct 2020 00:00:00 +0000

It’s finally here! An official supported version of VS Code that runs on a Raspberry Pi!

ICYMI: VS Code now officially supports @Raspberry_Pi too! 🍰

➡️ Download the Linux ARM and ARM64 builds here: https://t.co/aMc3pDFkDu pic.twitter.com/gsATu20K4X
— Visual Studio Code (@code) October 11, 2020

This to me is great news. The Raspberry Pi is a low-priced, small form factor computer that can run a full version of Linux. It’s popular with hobbyists and kids - it was originally designed to be a cheap computer for kids to learn to code on. It has the same standard USB and HDMI ports that a PC or Mac would have, as well as GPIO (General Purpose Input Output) pins that can be used to work with a wide array of external electronic components, devices, sensors, machinery and robotics.

What this VS Code release means is kids who are using a Pi can now use the same IDE that their grown ups use at work - Mum codes C# in VS Code at work and daughter codes Python in VS Code on a $35 computer at home connected to the family TV.

Lets look at how to get it set up.

Installing VS Code on a Raspberry Pi

STOP THE PRESS

VS Code is now in the Raspberry Pi apt repositories. You can install it by launching a terminal and running the following command:

sudo apt update
sudo apt install code -y

Using VS Code

Once the installer has finished, you will see Visual Studio Code as an option in the programming folder in the Pi menu. Select it to launch VS Code.

You can now install your favourite extensions and program away! Be aware that not all extensions will work fully!

Building smart garden ornaments with a Raspberry Pi and micro:bits

Sat, 11 Jul 2020 00:00:00 +0000

Ever wanted to track things happening in your garden, neighborhood or school using smart garden ornaments? Now you can using BBC micro:bits and a cloud-based IoT service.

TL;DR: check out this project on GitHub for a weeks worth of fun to do with families, friends or in the classroom using a Raspberry Pi, microbits and an IoT service.

github.com/jimbobbennett/smart-garden-ornaments

At Microsoft, we are celebrating July as our IoT month - after all who wouldn’t want to make use of the pun JulyOT! As part of this month, I created a fun project for schools, families and makers.

This is a fun week long project for those who want to take their first steps into the world of the Internet of Things (IoT) using devices that are popular with kids, and tools that make programming accessible to young developers. You’ll use a Raspberry Pi along with some BBC micro:bits and any garden ornaments you have to hand to build a smart neighborhood, gathering data such as temperature and noise levels and displaying it in the cloud using Azure IoT Central.

This is great for families, groups of friends, or even in the classroom - gathering data around your home, neighborhood or school.

Check out the project on GitHub! It’s broken down into 5 days of activities, from setting up the cloud service, to gathering data from micro:bits, to creating a mesh network to extend the range, to adding more sensors.

github.com/jimbobbennett/smart-garden-ornaments

I’d love you to contribute to this project! Please send in your stories of what you have set up, any additional ideas for ways to extend the project, or any other sensors you’ve set up!

Happy making!

Raspberry Pi cloud-connected Hat Cam

Thu, 09 Jul 2020 17:45:27 +0000

Build a cloud-connected hat/clothing camera powered by a Raspberry Pi and Azure!

This project builds a wearable, cloud connected camera. This is useful in situations where you want a record of what the camera sees stored in the cloud as quickly as possible. Make sure you use this in compliance will all applicable laws and privacy considerations.

This is video only, no audio. If you want to add audio, then I’d love a PR to add, or wait till I get a chance to add it.

This project requires a Raspberry Pi connected to a Raspberry Pi Camera and WiFi. It continuously records short videos from the camera (by default 10 seconds in length), and uploads these to Azure blob storage. The ideal setup it a Pi Zero W due to it’s small size and low power requirements, tethered to a mobile phone or hotspot for internet access.

This uses a cloud service to store the videos, and therefor may incur a cost. If you have a free trial account, you can get $200 for the first 30 days which should more than cover the cost. Check out the Blob storage pricing guide for more information on the cost of the service used.

This method allow resiliency - if WiFi connection is lost the upload will keep retrying. If the device is damaged then everything already uploaded will be secure. Files are stored on the SD card and only deleted after successfully being uploaded, so if there is no internet connection, files can be grabbed off the SD card.

This also allows audit trails - each short video is named with the current timestamp, and Blob storage stores the time the file was uploaded.

This project also includes a utility to download all the blobs and concatenate them into a single video file.

What’s required

To make this, you will need:

A Raspberry Pi Zero W (must be the W with WiFi access)
A Raspberry Pi camera module and appropriate ribbon cable that is as long as possible. The V2 camera is the best as it is small with good quality. The HQ camera can be used but is bigger which makes it harder to mount on clothing.
An SD Card
A USB power pack and appropriate cable to power the Pi
A mobile phone to tether to
A hat or other piece of clothing
Sewing materials, duct tape, bits of cloth to help affix the camera and Pi to your clothes

Set up the application

Before you can run the application, you need an Azure subscription and a storage account.

Set up your Azure Subscription

If you are a student aged 18 and up and have an email address from an academic institution, you can sign up for the free Azure for Students offer at azure.microsoft.com/free/students without a credit card. At the time of writing this gives you $100 of credit to use over 12 months, as well as free tiers of a number of services for that 12 months. At the end of the 12 months, if you are still a student you can renew and get another $100 in credit and 12 months of free services.
If you are not a student, you can sign up at azure.microsoft.com/free. You will need a credit card for verification purposes only, you will not be billed unless you decide to upgrade your account to a paid offering. At the time of writing the free account will give you US$200 of free credit to spend on what you like in the first 30 days, 12 months of free services, plus a load of services that have tiers that are always free.

Set up the storage account

An Azure storage account is a general purpose account to store data as files, queues, tables or blobs. In this app, each short video is stored as a blob in the storage account.

To create a storage account:

Head to ms.portal.azure.com/#create/Microsoft.StorageAccount-ARM to go straight to the create a new storage resource blade
Sign in with your Azure account if necessary
Select your Azure subscription
For the Resource group, select Create new. Name the resource group hatcam then select OK
Name the storage account. This name has to be globally unique across all storage accounts in Azure - this is because this name becomes part of the URL that can be used to access it. Pick a name that makes sense to you with something unique like your name, the date, etc.

This name can only contain lowercase numbers and letters
For the location, select the one closest to where you will be using the camera.

If you want to see the locations of the regions, check out the map in the Azure global infrastructure guide.
Leave the rest of the values as the defaults
Select Review + create
The storage account settings will be validated. Once done, select Create.

The deployment will begin. You will be notified when it’s complete.

Get the connection string

To connect to the storage account, you will need a connection string. To get this:

Open the storage account in the Azure portal. You can do this by selecting Go to resource from the notification that pops up with the storage account is deployed.
From the side bar, select Settings -> Access keys
Select the Copy to clipboard button next to the Connection string under either Key 1 or Key 2. It doesn’t matter which one.
Store this connection string somewhere. You’ll need it to configure your Pi code.

Deploy the code

Once you have the storage account, it’s time to deploy the code.

You can find the code in GitHub at github.com/jimbobbennett/pi-hat-cam.

The code is fully documented and commented, so check out the app.py file to see how it all works.

Configure the Pi

You will need to be running the latest Raspberry Pi OS on your Pi Zero W. To make the camera faster to launch, you should install Raspberry Pi OS Lite.

To do this, use the Raspberry Pi imager. When you choose the OS, select Raspberry Pi OS (other), then select Raspberry Pi OS Lite (32 bit).

Once your SD card is ready, configure the Pi for headless access by following the steps in this headless setup guide. Work through the steps up to the Remote desktop section - you don’t need to follow this section as it contains instructions when using the full Raspberry Pi OS, not the Lite version you are using here.

Deploy the Python code

The Python code you need is in this repo in the pi-app folder.

SSH into your Pi

Install git, Pip and Python virtual environments using the following command:

sudo apt install git
sudo apt install python3-pip
sudo apt install python3-venv

Clone the repo:

git clone git@github.com:jimbobbennett/pi-hat-cam.git

Navigate to the folder containing the code
```
cd pi-hat-cam/pi-app
```
Create a Python Virtual Environment:
```
python3 -m venv .venv
```
Activate the virtual environment:
```
source ./.venv/bin/activate
```
Install the Pip packages
```
pip install -r requirements.txt
```

Configure the app

Once the code is deployed, you’ll need to configure the app to connect to your Azure storage account.

The code uses the dotenv package to read environment variables from a .env file. You will need to create this file and add the Azure storage connection string that you copied earlier to it.

Create the file and open it for editing in nano:
```
nano .env
```
Add the following entry:
```
BLOB_CONNECTION_STRING=<connection_string>
```
Replace <connection_string> with the connection string you copied earlier.
Close the file by pressing ctrl+X. You will be prompted to save the file, so select yes and press enter to select the .env filename.

There are other optional parameters you can add to this file to configure things like the video resolution or quality. Check out the app.py code for more details.

Test the app

Before you use this in the wild, you should test the app.

Connect the camera to the Pi
Run the Pi code with this command:
```
python app.py
```

The app will start up and begin recording, uploading videos to blob storage once they are recorded. You should see something like this in the output:

(.venv) pi@hat-cam:~/pi-hat-cam $ python app.py
Checking container videos
Setting camera resolution to 1280 , 720
Starting queue worker
Recording ./videos/2020-07-08 07:42:51.h264
Queuing ./videos/2020-07-08 07:42:51.h264
Recording ./videos/2020-07-08 07:43:01.h264
Uploading ./videos/2020-07-08 07:42:51.h264 to 2020-07-08 07:42:51.h264
Deleting ./videos/2020-07-08 07:42:51.h264

Stop the app using ctrl+c
Head to the storage account in the Azure portal.
Select Storage Explorer from the side menu
You should see the blob files that were uploaded listed here

You can download these files by selecting one and selecting Download. The downloaded file can be opened in an app like VLC.

Configure the app to run on startup

Once the app is deployed, it can be run manually from the command line. It’s better, however, if it is started when the Pi is booted - that way it is running automatically not long after the Pi is connected to power.

You can do this by adding an entry to the crontab. Cron is a tool that runs code at certain times, and can be configured to run the Pi code when the Pi is rebooted.

Run the following code to configure the CronTab:

sudo crontab -l > cron.tmp
echo "@reboot sleep 30 && cd /home/pi/pi-hat-cam/pi-app && /home/pi/pi-hat-cam/pi-app/.venv/bin/python /home/pi/pi-hat-cam/pi-app/app.py" >> cron.tmp
sudo crontab cron.tmp
rm cron.tmp

This code writes a new entry that is run on reboot. First it sleeps for 30 seconds to ensure the Pi is fully running and connected to the internet, then it launches the app.py file using the Python binary in the virtual environment.

Test this out by rebooting the Pi. You should see video files appear in the Storage Explorer in the Azure portal. Power the Pi off when done.

Download the video

By creating short videos it helps ensure the videos are uploaded, but this is not the best for watching later. It would be easier if there was a way to reassemble the video. The video_downloader.py file in the downloader folder has code to do this.

This file needs a .env file set up the same as the main app, with the BLOB_CONNECTION_STRING value set. There are other values you can configure from the default, and this are listed in the code.

Ideally you should run this code on a device that is not the Pi, so that you can view the video. You will need Python installed.

From the folder the video_downloader.py is in, create a Python Virtual Environment:
```
python3 -m venv .venv
```
On Windows, use:
```
python -m venv .venv
```
Activate the virtual environment:
```
source ./.venv/bin/activate
```
Install the Pip packages
```
pip install -r requirements.txt
```
Create a file called .env in your favorite text editor, such as Visual Studio Code
Add the following entry:
```
BLOB_CONNECTION_STRING=<connection_string>
```
Replace <connection_string> with the connection string you copied earlier.
Run the code:
```
python video_downloader.py
```

This will download all the blobs, concatenating them into a single file called downloaded_video.h264. You can then view ths file in an app like VLC.

The blobs are not deleted when you download them. You will need to manually delete them from the storage resource once you are finished with them.

Create the wearable

There’s so many different pieces of clothing you could attach this to. In this guide I’m going to focus on a cap.

You will need:

A cap that you don’t mind making holes in
Scissors
Gaffer tape, or offcuts of fabric and needle and thread depending on how good you are at sewing (I’m not very good)
Double sided mounting tape
A USB power pack and USB cable that can reach from the pack in a pocket or strapped to a belt to the hat

Mount the camera

Make a small hole in the front of the cap - big enough for the camera lens to fit through
Place the camera inside the hat and push the lens through the hole. It might be easier to detach the camera from the board to pass it through the hole.
Fix the camera in place on the front of the hat with double-sided mounting tape. You can use fabric glue, but be careful not to get any on the lens. This also reduces you ability to easily re-use the camera in a different make.
Fix the camera circuit board in place inside the cap using gaffer tape
Fix the camera ribbon cable to the inside of the cap, right through to the hole at the back. Use gaffer tape or sew it in place.
Mount the Raspberry Pi Zero W to the back of the cap, attaching the camera ribbon cable. Use double sided mounting tape, gaffer tape or sew it in place, whatever works. The USB sockets should point downwards.
Test out the connections - power the Pi over USB and make sure videos are being uploaded. Also test out the fit and camera position, adjust it as necessary using tape or fabric.

Use the camera

To use the camera, first it needs to be tethered to your phone or a WiFi hotspot to upload the videos. To tether:

Connect to the Pi over SSH
Start your WiFi hotspot, or enable tethering on your phone
From your Pi, launch the configuration tool:
```
sudo raspi-config
```
Select 2 Network options
Select N2 Wireless LAN
Enter the SSID and Password of your hotspot
Select Finish
When asked to reboot, select Yes

When the Pi reboots, it will connect to your hotspot or phone.

Power off the camera until you are ready to use it.

Once you are ready to use it, connect the power and it will start recording as soon as it has booted up.

Here’s the camera in action browsing this repo:

Recognizing words on a microcontroller using TinyML

Tue, 21 Apr 2020 18:00:00 +0000

AI has traditionally been the realm of expensive computers, with models trained and run on GPUs and other expensive hardware. More recently this has started to change with a move towards TinyML - small machine learning models that are trained on the expensive hardware, but run on smaller, cheaper devices including the low power micro-controllers that run IoT and maker devices.

Adafruit have recently released a device to run TinyML models on called the EdgeBadge.

This device is based off their PyBadge device - a board with a screen, game style controllers, NeoPixel LEDs and connectors so you can wear it on a lanyard. The difference with the EdgeBadge is it has a built in microphone so you can use it for audio recognition. The product page has links to a demo you can run that recognizes two words, yes or no, and displays an image on screen when the words are detected.

You can find the demo on Adafruit Learn.

I thought it would be fun to repurpose this example and learn how to retrain the board to recognize ‘Stop’ and ‘Go’ instead of yes and no.

Train the model

The yes/no detection model is a TensorFlow model, created using an example from the TensorFlow GitHub repository. The model is trained using a huge file of recordings of people saying a number of different words. Once the model has been trained, it is then shrunk down to run on tiny devices such as micro-controllers, and weighs in at only a few kilobytes.

You can find the original TensorFlow example in the TensorFlow GitHub repo.

I trained my new model using the Azure Machine Learning Studio, a service that allows you to build and train models using cloud compute, then either host them in the cloud or download them for use offline.

Create an Azure account

To use Azure services you will need an Azure subscription. If you don’t have a subscription you can sign up for free.

If you are a student aged 18 and up and have an email address from an academic institution, you can sign up for the free Azure for Students offer at azure.microsoft.com/free/students without a credit card. At the time of writing this gives you $100 of credit to use over 12 months, as well as free tiers of a number of services for that 2 months. At the end of the 12 months, if you are still a student you can renew and get another $100 in credit and 12 months of free services.
If you are not a student, you can sign up at azure.microsoft.com/free. You will need a credit card for verification purposes only, you will not be billed unless you decide to upgrade your account to a paid offering. At the time of writing the free account will give you US$200 of free credit to spend on what you like in the first 30 days, 12 months of free services, plus a load of services that have tiers that are always free.

Create a Machine Learning resource

Before you can use the Azure Machine Learning (ML) Studio, you need to spin up a Machine Learning resource. This is created in your Azure subscription but then can be shared with other developers so that you can all share the same ML Studio workspace.

Open the Azure portal
Select Create a resource then search for Machine Learning. Select Machine Learning then select Create.
Give the workspace a name and select your Azure subscription.
Create a new resource group for this service.

Resource groups are logic groupings of services, so it is good practice to put all the resources for one project in a single resource group so that you can manage them together. When you create a Machine Learning resource it will create a few different services, so having them in the same resource group means you can delete them all together by deleting the resource group.
Select the location nearest to you. For the workspace edition, select Basic. This price tier of the core service is free to use, you only pay for the compute used to train or run models.
Select Review + Create, then select Create

The resource will be created.

Set up ML Studio

Now that the Machine Learning resource has been created, you can use it from ML Studio.

Open ML Studio at ml.azure.com
Log in with your Azure account, then select your directory, subscription and the workspace you just created
Select Get started

ML Studio will log in to your workspace. From here you can run Machine Learning workloads using Jupyter notebooks, AutoML to create models for you based off data sets, or a drag and drop designer.

Set up compute

The TensorFlow example uses a Jupyter notebook to train the model. For this, I used a slightly tweaked version of the notebook, tweaked to fix some versioning issues and use the power of ML Studio.

To run a notebook, you need compute - a machine allocated to run the training. This compute is paid for based on the time it’s running and the power of the machine, the more powerful, the more you pay per minute.

You can see the pricing per hour on the Machine Learning pricing page

Select Compute from the left-hand menu
Select + New
Give the compute a name
Drop down the Virtual Machine Size box, and select GPU to list the GPU-based compute options. Select the Standard_NC12 machine - this has 2 GPUs and is enough to train this model in an hour and a half or so. At the time of writing the cost is US$1.80 per hour when running in the US.
Select Create

The compute will be created a spun up.

Upload the notebook

You can find the notebook in the Releases in this GitHub repository. Use this link to download the notebook:

word-recognizer-training.ipynb.

Once you’ve downloaded the notebook, you need to upload it to ML Studio.

Select Notebooks from the left-hand menu
Select Upload files
Find the notebook you’ve just downloaded, and select it

The notebook will be uploaded and available to run. You will see it in a file tree on the left hand side of the Notebooks blade.

Select the notebook and it will open up in read only mode
Select the compute you created in the menu at the top
Select Edit -> Edit in Jupyter top open the notebook in Jupyter notebooks. You will need to sign in to your Azure account again.

The notebook has a number of sections, each documented so read the notebook to see what each section does. A basic overview is:

Create some environment variables to define the words to train for and the number of training iterations.
Install a specific version of TensorFlow GPU to ensure the training uses the GPU, and uses a version that the example training script was written for

The training script will only work against TensorFlow 1.5.x, don’t update the notebook to a later version otherwise the training will fail
Clone the TensorFlow repository to get access to the training script
Run the training, then freeze the created TensorFlow graph to save down the weights
Convert the model to TensorFlow light to make it smaller, then save it out as binary data in a C code file so that it can be used from code.

From the Jupyter notebooks:

The first cell defines the words the model should be trained for.
```
# A comma-delimited list of the words you want to train for.
# All other words will be used to train an "unknown" category.
os.environ["WANTED_WORDS"] = "stop,go"
```
The data set supports the following words:

Yes No Up Down

Left Right On Off

Stop Go Zero One

Two Three Four Five

Six Seven Eight Nine

The notebook is set to train for ‘Stop’ and ‘Go’, but if you want to change the words it is trained for, change the array.
Select Run from the top menu for each cell to run them one at a time, or select Cell -> Run All to run all the cells. The training step will take 1.5-2 hours to run.
Once the steps have all finished, head back to ML Studio, select Compute from the left-hand menu, check the box next to your compute and select Stop to stop the compute.
Select Notebooks from the left-hand menu. Expand the Content node above the notebook in the file explorer, and select the tiny_conv.cc file. This is the C code for the resulting model.


Yes	No	Up	Down
Left	Right	On	Off
Stop	Go	Zero	One
Two	Three	Four	Five
Six	Seven	Eight	Nine

Don’t forget to stop the compute once the model is trained! You pay for the time that the compute is running, not the time that the notebooks are running, so to avoid unnecessary costs, remember to stop the compute as soon as you are finished, then restart it if you want to train the model again.

Compile the code for the EdgeBadge

Now that the model is created, it can be compiled into the sample app for the EdgeBadge.

Set up the development environment

It takes a bit of setup to be able to program Adafruit devices using C++. The Arduino IDE is the main tool for programming them, but I personally like to use Visual Studio Code as an IDE, and the Arduino extension to allow Visual Studio Code to control Arduino to program the boards.

Install the tools

Install Visual Studio Code
Install the Arduino IDE - NOTE On Windows DO NOT install using the Windows store, instead use the Windows Installer, for Windows XP and up.
Inside Visual Studio Code, select the Extensions tab from the left hand side, and search for an install the Arduino extension. Make sure to install the one from Microsoft as there is more than one extension called Arduino.

Configure the Arduino IDE and Visual Studio Code for Adafruit development

To talk to Adafruit boards, the Arduino IDE needs to be configured to know how to send the compiled code.

Follow the instructions on the Adafruit site to configure the Arduino IDE for Adafruit boards: learn.adafruit.com/adafruit-pybadge/using-with-arduino-ide
From Visual Studio Code launch the command palette, then select Arduino: Library manager. From here you can install some additional libraries that are needed to compile the code.
Search for and install the following libraries:
1. Adafruit EPD
2. Adafruit TinyUSB Library
3. Adafruit WavePlayer Library
4. Adafruit SPIFlash
5. Adafruit ImageReader Library
6. Adafruit ST7735_and_ST7789 Library
7. Adafruit Unified_Sensor
8. Adafruit LIS3DH
9. Adafruit TouchScreen
10. Adafruit NeoPixel
11. Adafruit GFX Library
12. Adafruit Arcada Library
13. Adafruit Zero_PDM Library
14. Adafruit ZeroTimer Library
15. Adafruit TensorFlow Lite
16. Arduino TensorFlowLite
17. SdFat - Adafruit Fork (make sure to install the Adafruit fork, not the original SdFat)

Configure the board

Connect your EdgeBadge to your computer via USB and make sure the device is switched on
From the command palette, select Arduino: Board Configuration

Select the following settings:

Setting	Value
Selected board	Adafruit pyBadge M4 Express (SAMD51)
Cache	Enabled
CPU Speed	180 MHz (overclock)
Optimize	Fastest (-Ofast)
Max QSPI	50 MHz (standard)
USB Stack	TinyUSB
Debug	Off

From the command palette, select Arduino: Select serial port and select the port the EdgeBadge is plugged into. On macOS it will be named something like /dev/cu.usbmodem<xxx> where <xxx> is a number. On Windows it will be called COM<x> where <x> is a number.

Compile the code

The code for the edge badge is in the word_recognizer folder in this GitHub repo. Clone the repo and open the word_recognizer folder in Visual Studio Code.

The code contains images and a model that can recognize two words, ‘Stop’ and ‘Go’.

Double tap the Reset button on the back of the EdgeBadge to put it into the boot loader ready to program
From the command palette, select Arduino: Upload. The code will be compiled and uploaded to the board.
Once the device reboots, say the words ‘Stop’ and ‘Go’, and the board will show an image for the word you have said.

Select the image below to see a video of this in action.

Use your own model

If you want to use your own model trained using Azure ML Studio, you will need to update the code to use this model, and provide image files.

Upload image files

Create bitmaps for each word that you are detecting, named in the format <word>.bmp. For example, if you trained the model for ‘Yes’ and ‘No’ you will need Yes.bmp and No.bmp. These files must be 24-bit bitmaps at a size of 160x128. The best tools to use to create these is good old-fashioned MS Paint on Windows or ImageMagik on macOS.
Reset the EdgeBadge. It will mount on your computer with a disk called CIRCUITPY
Copy the bitmaps into this drive

Update the code

Open micro_features_micro_model_settings.cpp
Update kCategoryLabels to include the words the model was trained on. Keep "silence" and "unknown" at the start of the array, and put the words after in the order they were specified in the notebook. Ensure the casing of the words matches the names of the bitmaps created earlier.

The words HAVE to be in the same order as the notebook or the wrong word will be detected
Open `micro_features_micro_model_settings.h
Update kCategoryCount to be 2 higher than the number of words you trained the model for, so that it is the same as the size of the kCategoryLabels array. For example if you trained it for ‘Yes’ and ‘No’, set kCategoryCount to 4.
Open micro_features_tiny_conv_micro_features_model_data.cpp
Replace the contents of the g_tiny_conv_micro_features_model_data array with the values from __content_tiny_conv_tflite in the tiny_conv.cc file downloaded from the Azure ML Studio notebook
Replace the value of g_tiny_conv_micro_features_model_data_len with the value of __content_tiny_conv_tflite_len from the tiny_conv.cc file downloaded from the Azure ML Studio notebook
Use the steps above to compile and deploy this new code to the device, then test it out.

Let your family know you are in meetings with an IoT Busy light

Thu, 19 Mar 2020 00:53:47 +0000

Like a lot of folks at the moment, I’m working for home and my child is off school.

For those reading this in the future and wondering why, I am currently living in the time of the COVID-19 pandemic and worldwide toilet paper shortages.

One of the upsides of working from home, especially when my 7 year old is off school is that I am available most of the time for the occasional quick cuddle, to laugh at something silly she’d just done or help her with something. One of the downsides is that at any time she could walk in to my office whilst I’m on a call.

What I need is a on-air style light, to let her know when I’m in meetings and when I’m not. Luckily I’m an IoT nut so I not only have the skills to build such a thing, but I have all the parts just lying around begging me to use them! So I built one!

What I needed it to do

This light needs to live outside my office so my family can see it before they come in, but I don’t want to have to get up and walk out the door to turn it on or off. I want to have it controlled automatically by my calendar, so it shows red when I have a meeting and green when I’m free.

This leads to the problem of connectivity - how can my calendar control my light remotely. For that the answer was obvious - a mixture of Azure IoT Central, the glorious IoT SaaS platform from Microsoft, and Azure Logic Apps, a no-code tool for building apps. Both have such a low barrier to entry I knew I’d be able to get something up and running quickly.

Hardware

All the Amazon links below are affiliate links. Any money I make from these links will be donated to support families affected by COVID-19. If you don’t want to use these links then search for the items directly on Amazon or buy from your local maker store or direct from the manufacturers.

Every good maker project starts with hardware. I needed a device I could program, some lights and something to put it in. After digging around in my supply of bits I put together the following:

Pimoroni Mood Light - this is a nice little kit that contains a Raspberry Pi Zero W, a Pimoroni unicorn pHAT with 55 programmable LEDs in a rectangle, some laser cut plastic to make a case that looks like a light, and various cables and headers.
16GB SD Card - the mood light doesn’t come with an SD Card, so you’ll need one to run the code from.

The Pimoroni mood light is a great all-in-one package for a light, but any Raspberry Pi would work as long as it has internet connectivity. If you want to get the Pi parts separately and make your own enclosure, then you will need:

Raspberry Pi Zero W - the W part is important as it needs internet connectivity
Headers for the Pi - the kit linked above includes them, but if you buy a board on it’s own you will need the headers
Pimoroni Unicorn pHAT - this is a grid of 55 programmable LEDs that you can control from the Pi

One of the downsides to a make like this is it involves the bane of my existence - a soldering iron…

The Pi Zero doesn’t come with headers pre-soldered unless you are lucky enough to track down one that does. The Unicorn pHAT also needs to be soldered. If you don’t have a soldering iron, nows the time to buy one and learn. Be warned, the headers are small and fiddly, so you may also need a magnifier to help you.

If you are new to soldering, some great videos to help you are from EEVblog:

You can also use a different Pi such as the new Raspberry Pi 4, and these have the headers already on the board. There are also many different light options you could use, so work with whatever you have, but you’ll have to work out the code to use them yourself.

Configure IoT Central

My favorite IoT cloud service is Azure IoT Central. This is a software as a service (SaaS) platform that allows devices to send data to and be controlled from the cloud. I can use this to send commands to the device to change the color of the light. I can also use to cache the current color so that if the device resets, it can retain it’s previous color setting.

Set up the IoT Central app

Head to aka.ms/BusyLightTemplate. This link is to a pre-configured application template that you can use to create the IoT Central application
You may be prompted to sign in, so sign in with your Microsoft account. If you don’t have one you can create one during this step. A Microsoft account can be created with an email address from any provider, it doesn’t just have to be hotmail or outlook.
Give the application a name and a URL. The URL will need to be globally unique across all IoT central apps so add something personal or leave the auto-generated name.
Select a pricing plan. You can have a free plan for 7 days without an Azure account, or a standard plan using an Azure account. Every account has 2 devices for free, so as long as you are only using 1 or 2 devices then you will never have to pay.

You will need an Azure subscription for a later step, and you can sign up for free:
- If you are a student aged 18 and up and have an email address from an academic institution, you can sign up for the free Azure for Students offer at azure.microsoft.com/free/students without a credit card. At the time of writing this gives you $100 of credit to use over 12 months, as well as free tiers of a number of services for that 2 months. At the end of the 12 months, if you are still a student you can renew and get another $100 in credit and 12 months of free services.
- If you are not a student, you can sign up at azure.microsoft.com/free. You will need a credit card for verification purposes only, you will not be billed unless you decide to upgrade your account to a paid offering. At the time of writing the free account will give you US$200 of free credit to spend on what you like in the first 30 days, 12 months of free services, plus a load of services that have tiers that are always free.
If you go with a standard plan, select your Directory, Azure subscription and the Location nearest to you.

If you go with the free plan, enter your contact information.
Select Create

The IoT Central app will be provisioned and the dashboard will be loaded.

Create a device

The app that is created contains everything you need to get going - except a configured device. From this app you can configure one or more devices and control these.

Select Devices from the left-hand menu
Select Busy Light from the Devices list
Select + New from the top menu
Give the device a name and an Id, then select Create
Select the newly created device, then select Connect from the top menu. This will show some connection values.
Take a note of the ID scope, Device ID and Primary key

Repeat this for as many devices as you want to control.

Program the Pi

The device will be running some Python code that can control the Unicorn pHAT as well as talk to IoT Central.

Set up the Pi

The Pi needs to be running Raspbian Lite

Configure Rasbian Lite on the SD card by following instructions here: raspberrypi.org/documentation/installation/installing-images
Once the image has been burned, open it in your file explorer or Finder. It will be mounted as a drive called BOOT. You may need to eject the re-insert the SD card to make it visible after burning.
Create an empty file in the root of the drive called ssh with no extension. This file should be empty, the presence of a file called ssh is enough to enable SSH on the Pi.
Create a new file in the root of the drive called wpa_supplicant.conf. This file contains the WiFi connection details. Add the following to this file:
```
ctrl_interface=DIR=/var/run/wpa_supplicant GROUP=netdev
update_config=1
country=US

network={
 ssid="YOURSSID"
 psk="YOURPASSWORD"
 scan_ssid=1
}
```
Replace YOURSSID and YOURPASSWORD with the SSID and password of your WiFi.
Eject the SD card and insert it into your Raspberry Pi.
Power up the Pi. It should only take a few seconds to boot, and whilst this is happening you will see the LED by the power socket flash.
Test the connection by SSHing into your Pi. From your computer connect to pi@raspberrypi.local.
```
ssh pi@raspberrypi.local
```
If the host cannot be found then if you know the IP address (for example by using your routers management software) then you can log in to pi@192.168.0.1, replacing 192.168.0.1 with the IP address of your Pi. You can also use the Adafruit Pi Finder utility to locate it’s IP address.

The default password is raspberry.

Install the light controller software

The light controller software is in my GitHub repo in the app.py file. This contains code to talk to IoT Central to receive commands to change the light color, then set this color on the Unicorn pHAT. For this script to connect to IoT central some environment variables need to be configured in a file.

Run the install script using this command:
```
bash -c "$(curl -s -L https://github.com/jimbobbennett/BusyLight/releases/download/v0.1/install.sh)"
```
This will update the Pi, install libraries for the Unicorn pHAT, the Python code for this app, and configure the CronTab to run this app when the device reboots. You may need to select ‘y’ one or more times when prompted.
When prompted enter the values for the ID scope, Device ID and Primary Key copied from the IoT Central device connection dialog
The app will be installed in the BusyLight folder in the home directory
After installation, the device will reboot and you will be disconnected from the SSH session. When the Pi reboots the app will be running.

If you are using a different light than the Unicorn pHAT you will need to install the relevant libraries yourself and update the code. The code is in the ~/BusyLight/app.py file.

You will need to remove all references to the unicornhat module and the calls to set it up. Then update the set_color function to use your preferred lights. This function is passed the color as an RGB HEX string, the first 2 characters are the hex value to set for the red channel, second two are the hex value for the green channel and last two the hex values for the blue channel.

Test the light

The light is now ready to be controlled from IoT Central.

Head to your IoT Central app in your browser
Select Devices from the left-hand menu, then select your device
You will see a dashboard with 2 cells - one showing the current color, then other labelled Update Colour. Select the Update Colour cell.
Enter a value for the Colour using a 6-character HEX string for RGB values. For example, for red enter FF000, green is 00FF00, blue is 0000FF, white is FFFFFF and off is 000000
Select Run

The light should change to match the color specific. Try with a few different colours.

If you return to the dashboard, you will see the value of the Colour cell updated to show the value sent to the light. This value is read by the app every time it restarts to ensure that the light keeps its color between restarts.

Connect my calendar to control the device

Finally my light needs to be controlled. I want it to turn red automatically when a meeting starts, then turn green when it ends. This can be done using Azure Logic Apps.

Azure logic apps are not free, but are VERY cheap for small numbers of runs. For example at the time of writing, a single run on a logic app is $0.000025 per action, so to poll for meetings every 10 minutes each day might cost $0.50 a month.

In this I cover using Logic Apps to connect to my Office 365 calendar, but you can use these to control IoT Central triggered by a number of different things such as:

A web request - this can then be called by a device such as a Flic button

A timer to turn the light on and off based of a normal work day

Azure dev ops - turn the light red when a work item is assigned

Create the logic app

Open the Azure Portal at portal.azure.com
Log in if you need to using your Microsoft account. If you don’t have an Azure account follow the instructions above to create one.
Select + Create a resource
Search for Logic App, select it then select Create
Select your Azure subscription
For the Resource group select Create new and name the new resource group BusyLight

Resource groups are logical groupings of Azure services, allowing you to manage all the services for a particular application or project together.
Name the logic app CalendarControl
Select the location closest to you
Select Review + create, then Create

The logic app will be created, and when it is ready select Go to resource

Program the logic app

Logic apps are no-code apps where you can connect triggers to actions, and these can connect to external services such as Office365 and IoT Central. I want mine connected to my Office365 calendar to poll for upcoming events, and if it finds any schedule calls to IoT Central to execute the Update Colour command.

Create a trigger

When you first open the Logic App it will take you to the designer. From the Templates section, select Blank logic app
Search the connectors and triggers for office 365 outlook and select the Office 365 Outlook connector
Select the When an upcoming event is starting soon trigger
The designer will show the new trigger. Select Sign In and sign in with your Office 365 account.
Select your calendar in the Calendar Id drop down

If you have multiple calendars to check, you will need to create multiple Logic Apps.
Leave the look ahead time at 15 minutes
Set an appropriate interval to poll for new items. Depending on how far in advance meetings can added to your calendar you can choose a suitable interval, as you pay for each poll. I use 10 minutes as I don’t get meetings appearing in my calendar with less notice than that.

This value needs to be less than the look ahead time otherwise events might be missed

Wait till the event starts

The trigger is fired when it sees an upcoming event, so the app needs to delay till the event actually starts before changing the light to red. Logic app triggers and actions can pass data between them, so the Office 365 connector can pass the start and end times to the next action in the app.

There is a small snag however - date and time formats are hard! The date format passed from the Office connector is not quite right for the one expected by the delay actions, so needs to be adjusted.

Select + New step
Search for Schedule and select Schedule
Select the Delay until action
Select the Timestamp value and a box should pop up allowing you to build an expression
Select the Expression tab, then select Concat
Select the Dynamic content tab and select Start time. If you can’t see this, select See more from the top of the list.
Select the expression, and add , 'Z' before the last bracket. The select OK.

Execute the IoT Central command to change the light color

The delay will hold till the meeting is about to start, so the next step is to connect to IoT Central to execute the command. To do this, you will need the IoT Central application Id and device Id.

Open your IoT Central app
Select Administration from the left-hand menu
Select Your application from the Administration menu
Take a note of the Application ID
Select Devices from the left-hand menu
Take a note of the Device Id of your device

Once you have this Id, you can create the connector

Head back to your logic app
Select + New step
Search for Azure IoT Central, then select Azure IoT Central
Select the Run a command action
Once the action appears, select Sign in and log in with your Azure account if required
Drop down the Application box and select Enter custom value
Enter in the Application ID you noted down earlier
Drop down the Device Template and select Busy Light (1.0.0)
Enter your device id in the Device box
Drop down the Command and select Update Colour
Drop down Add a new parameter and check the box next to Colour
Set the value of Colour to be the colour you want when you are in a meeting, using a 6 character RGB hex string. For example, for red enter FF0000

Handle the event ending

Repeat the two steps above to add another Delay Until and Run a Command action to the app. The second delay should use the End time instead of the Start time, and for the colour use the colour you want when you are not in a meeting, such as 00FF00 for green.

Save the logic app

Select Save from the Logic App designer menu to save the Logic App and enable it

Test the logic app

Add an upcoming event to your calendar - far enough away that the poll will be run once. Your light will turn red when the event starts and green when it finishes.

Next steps

I also added a Flic button to control the light, in case of impromptu meetings or if a meeting ends early (Yay!). To do this I set up three Logic Apps with HTTP triggers and actions calls the IoT Central command passing different colors - red, green and off (000000). I then connected these to the Flic button, so one click for red, two for green and hold to turn the light off.

Learn more

If you want to learn more about Azure IoT and AI services, the best place to start is Microsoft Learn - our free on-line, self guided hands on learning experience.

Setting an Azure IoT Hub connection string in code on an MXChip

Wed, 12 Feb 2020 02:02:48 +0000

I was recently asked if there was a way to set the Azure IoT Hub connection string for an MXChip board in code. Normally you’d push this to the EEPROM using the tooling in VS code, or from a terminal using SSH as described here. In this situation, this was for students and was needed for two reasons:

The students would be sharing lab PCs and MXChip boards so would need to constantly log in and out of the Azure extensions in VS Code and re-configure boards - something they would probably forget to do every now and again causing a support headache.
The lecturer would need to be able to take their code and run it on their own board to test, so would need their connection string. To save time configuring boards and looking up connection strings, it would be better to have it in code. Again, it could easily be forgotten!

NOTE - this is potentially a very bad thing as you can end up essentially putting secrets in code. DO NOT do this for public code, code then ends up on GitHub or anything like this, this only makes sense for private code submitted internally for something like a students assessment using a hub on a free tier so cannot cause any cost if it gets flooded.

Out of the box there are no APIs available to do this. However, there is a way!

When connecting to Azure IoT Hub over MQTT, you call DevKitMQTTClient_Init and this loads the connection string from EEPROM and uses it for the connection. As it turn out, as well as being able to read from EEPROM in code, you can also write to the EEPROM, meaning you can set the value before it is read.

Using this, it wasn’t too hard to write the code to set this value:

# include "EEPROMInterface.h"
# include "SerialLog.h"

...

void setup() {
 ...

 if (WiFi.begin() == WL_CONNECTED)
 {
 // Write the connection string to EEPROM as an array of uint8_t
 EEPROMInterface eeprom;
 char connString[] = "<my connection string>";
 int ret = eeprom.write((uint8_t*)connString,
 strlen(connString),
 AZ_IOT_HUB_ZONE_IDX);

 // Check the write worked - 0 means it was written
 // Less than 0 is an error
 if (ret < 0)
 {
 LogError("Unable to get the connection string from EEPROM.");
 return;
 }

 // Connect as normal, this will read the new value
 // for the connection string
 DevKitMQTTClient_Init();
 ...
}

Replace <my connection string> in the above code with your connection string. It will then be written to the EEPROM before the call to DevKitMQTTClient_Init.

If you read the EEPROM write documentation, you will see zones listed. These are defined areas in the EEPROM and you can use these to write the WiFi SSID and Password as well as the connection string. This is useful if you want to build a solution that downloads new WiFi details.

Storing IoT Data in a database

Thu, 05 Dec 2019 23:17:04 +0000

One question that comes up a lot when getting started with IoT is ‘How do I store all this data?’. This is an important question - IoT creates a LOT of data, some of which can be analyzed on the fly, but there are a large number of cases where you want to run analytics later, or have access to historical raw data to run new analytics later.

IoT typically involves ingesting a large number of messages through a single hub, and these hubs are typically dumb - in that they provide a single point to ingest the messages but don’t do anything with them. That’s the responsibility of the developers building out the IoT solution.

One area that is powerful with Azure IoT is Azure Stream Analytics. This is a service that can be connected to any event stream, such as an IoT Hub, and take the live stream of data and do something with it, such as filter, manipulate and output to other systems. You can even wire up ML based anomaly detection to be alerted when unexpected values are received.

In this post we will see how to use Stream Analytics to stream IoT data into a database. This assumes you already have an IoT hub receiving data.

Create a database

The first thing you need is a database to put the data into. Stream analytics works with both Azure Cosmos DB and Azure SQL Database, as well as many other storage solutions, so it’s up to you which one you want to use. In this post I’ll use Cosmos DB, but the concepts apply to all data stores.

Launch the Azure Portal
Select + Create a resource from the menu on the left or the dashboard
Search for Azure Cosmos DB, select it and select Create
Select your subscription, select or create a new resource group and name your account.
Set the API to Core (SQL). Stream Analytics only supports the SQL API.
Set the location closest to you, then select Review + create, then select Create
This will take a few minutes to create, so once done load it in the portal
Create a new collection to store the data. From the Cosmos DB resource, head to the Data Explorer tab
Select New Container
Give the database a name
Set the throughput. You can find a discussion around throughput in the documentation. For a simple test setup, set this to be 400 (the minimum).
Give the container a name
Set a partition key. You can read more on partition keys in the documentation
Select OK

Create a Stream Analytics resource

Once you have the database, you need to create a Stream Analytics resource.

Launch the Azure Portal
Select + Create a resource from the menu on the left or the dashboard
Search for Stream Analytics job, select it and select Create

Give the job a name, select your subscription, select or create a new resource group and set the location closest to you.
Stream Analytics jobs can be hosted in the cloud, or run on the edge via an IoT Edge Gateway device. For now, select Cloud, but you can learn more about running on the edge in the documentation.
Set the number of Streaming units you need - the more data coming in, the larger the number of streaming units you need. You can read more about streaming units in the documentation, and learn about pricing on our pricing page. For now leave this as the default of 3.

Select Create.

Connect the dots

Stream analytics allows you to define inputs, outputs and queries. Queries query inputs and can push data into outputs, so you can query all items from an IoT Hub and select them into a Cosmos DB database. These queries run on the stream as data is received, and output as each item is processed - these are not like traditional SQL queries that run synchronously against a static set of data.

Adding the IoT Hub as an input

Select the stream analytics job in the Azure portal
Head to the Inputs tab
Select + Add stream input, then select IoT Hub

Give the input an alias - this is the name you will use in your queries
Ensure Select IoT Hub from your subscription is selected, and select your IoT Hub. Leave the rest of the values as the defaults.
Select Save

Adding Cosmos DB as an output

Head to the Outputs tab
Select + Add, then select Cosmos DB

Give the output an alias - this is the name you will use in your queries
Ensure Select Cosmos DB from your subscriptions is selected, and select your Cosmos DB account, database and container.
Set the Document id field to be the unique key field for the records you will be inserting. If you are inserting directly from IoT Hub without any translation then use what ever unique id is set on each message.
Select Save

Building the query

Head to the Query tab
Change the query to be:

SELECT
 *
 INTO
 [cosmos-db]
 FROM
 [iot-hub]

Set [cosmos-db] to be the alias you used for your Cosmos DB database. This needs to be inside square brackets, so if you used myCosmosDB as the alias, you would use [myCosmosDB].
Set [iot-hub] to be the alias of the IoT hub, again inside square brackets
Select Save query

Start the query

Head the the Overview tab
Select Start
The query will start listening to data coming in the input, and send it to the output

Once data comes in, you will be able to see it appear in Cosmos DB.

Learn more

Create a free Azure account
Check out the Stream Analytics documentation.
Browse the [Stream Analytics modules on Microsoft Learn](https://docs.microsoft.com/learn/browse/?term=stream analytics&WT.mc_id=streamanalytics-blog-jabenn) and grow your skills via hands on learning.

Connecting your Domain.com domain name to your web app during an MLH hackathon

Wed, 27 Nov 2019 01:55:32 +0000

You’re at an MLH Hackathon and you decide you want to win the Domain.com prize for the best domain name registered at the event. You’ve decided on the ultimate name, and now you want to use it for your hack…

But what do you do next? How can you take your amazing code and have it appear across the internet using your new domain name? Well this post is here to help!

Deploy your code to the cloud

The first thing to do is deploy your code to the cloud, ready to have the domain name assigned. To do this, you will need to deploy it to an Azure App Service.

If you don’t already have an Azure account, sign up for a free account now!

Students can sign up at aka.ms/FreeStudentAzure and get US$100 to use for 12 months, along with free tiers of a load of services for 12 months along with other services free for life, and after that 12 months is up, as long as you are still a student you can renew each year and get another US$100/12 months of free stuff. No credit card required.
If you are not a student, you can sign up at aka.ms/FreeAz and get US$200 to use in the first 30 days, as well as 12 months of free services along with other services free for life.

Next deploy your code to Azure App Service. App Service is a fully managed platform to run your web apps, be they Flask, Node, PHP, ASP .NET, whatever. You supply the code and App Service will host it for you. There are loads of ways to deploy to App Service, so I won’t cover them all here. You can read about deployments on the App Service docs. My personal favorite is the Azure App Service extension for VSCode, and the instructions for using it are in their GitHub ReadMe.

One thing to note - you cannot assign a custom domain to a Free Tier app service, you will need to use a higher tier, such as the B1 tier. At the time of writing in the US this is about US$13 a month, so don’t forget to delete this resource after the hackathon if you don’t want to keep being billed! It’s also free for the first 30 days, so you won’t need to pay anything during the weekend.

If you’ve already configured your App Service using the free tier, don’t worry - you can scale it up from the Azure Portal. All the instructions are in the App Service docs.

Configure your domain to point to your app

Once your app is deployed you need to point your new domain at it. This is a two step process - you need to tell Domain.com that you want to point your domain to your App Service, and then tell your App Service to handle requests at that domain.

Configuring Domain.com

The first thing to do is to get the IP address of your App Service. You can do this from the Azure Portal or the command line.

From the portal:

Log into the Azure Portal, and find your App Service resource.
Select the Custom domains menu item. From here, copy the IP address.

From the command line:

nslookup <your-app-service>.azurewebsites.net

Substitute <your-app-service> with the name of your App Service.

Next you need to configure domain.com to point to this IP address.

Select the DNS and Nameservers option from the menu on the left

Under the DNS Records section there will be a number of records. One will have a Record of A, a Name of @ and the Content set to an IP address. Click the three downwards ellipses at the end of the row and select Edit.

Set the IP address to be the IP address you copied from the Azure portal, then select Update DNS.

Look for a TXT record with a name of @. Edit this record and update the Content to be the URL of your App Service which will be <your-app-name>.azurewebsites.net.

Configuring App Service

Once Domain.com has been configured, you need to tell your App Service to use it. This can be done from the Azure Portal, or the cli.

From the Azure Portal

Select the Custom domains tab
Select + Add custom domain

In the blade that pops up, enter your domain name that you registered with Domain.com. Then select Validate. This will validate that the records are set up correctly and the hostname is available. If everything is valid you will see two green ticks at the bottom.

In theory the DNS record updates can take up to 48 hours to propagate, but in my experience it happens in a few minutes. If the validation fails then wait a minute and try again.

Once everything has been validated, select Add custom domain.

From the Azure CLI

Make sure you have the Azure CLI installed. If not, follow the installation instructions. Otherwise use the online cloud shell at shell.azure.com.
Enter the following command:

az webapp config hostname add --webapp-name <your-app-servce> --resource-group <your-resource-group> --hostname <domain-name>

Substitute <your-app-servce> with your app service name, <your-resource-group> with the name of the resource group your app service lives in, and <domain-name> with the domain name from Domain.com.

If you see any errors, wait a few minutes are try again as the DNS records may not have propagated.

Done!

That’s it - everything should now work. Head to http://<your-new-domain>.<tech/online/site> and you should see the app you deployed to App Service.

You may see a 404 or 403 error to start with - this is because DNS records are cached for a while to save lookups, and your browser is using an old record. To fix this you can either try a new browser or browser window (this sometimes works), or flush your DNS cache.

On Windows launch a command prompt and run:

ipconfig /flushdns

On macOS launch the Terminal and run:

sudo killall -HUP mDNSResponder;

Enter your password when asked.

On Linux this depends on which distro you are using, so it would be easier to search for the relevant instructions using Bing.

Good luck with your hackathon, and I hope you win!

Getting an internship at Microsoft

Thu, 17 Oct 2019 19:39:16 +0000

Come as you are, do what you love

I’ve been asked a few times for thoughts and tips around getting an internship at Microsoft. So in the spirit of “I didn’t have time to reply to you so I wrote this blog post”, I thought I’d provide a stack of resources useful to folks wanting to know more about the process.

Application

Interested in an internship here at Microsoft? The first port of call is the University internship Careers site. This is the US site, and we have similar sites globally so if you are looking outside the US then head to careers.microsoft.com and search for international internship opportunities.

This site has a wealth of information around the program, as well as a link to apply. Get those applications in now!

The interview process

This is a question I often get asked - what is the interview process? It varies depending on the type of role, and I’m not a recruiter so I can’t talk officially on the topic, but there are some great posts from former interns that might help shed some light on it.

Note that these are third party posts and have not been checked for accuracy or correctness. The content may be out of date.

Diary of an Ex-Microsoft Intern on Fast Company by Will Lawrence

How I got an internship at Microsoft — interview process and learnings by Aditi Khazanchi

microsoft-internships open document on GitHub by Courtney Thurston

How I made into Microsoft: The Off-campus way by Ronak Sakhuja focusing on the India intern recruitment process.

The main thing is to come as you are and do what you love - out mantra for recruiting. Let your passions shine through, talk about the projects you are most proud of, the work you do outside the classroom, hacks you attend. Be aware of Microsoft and its culture. And most importantly, be yourself.

Life as an intern

You’ve made it through the interview and have an internship. So what’s it like here?

Sneak Peak into the Life of a Microsoft Intern by Jacob Shiohira

12 Weeks at the Microsoft Garage by Dominic Whyte

It’s all happening here

Yes - it’s true, it is all happening here. Don’t believe me, then see what the last set of interns did in their spare time:

Let me know your experiences

I’d love to know your experiences - please add a comment with any links to posts you have written about your time as a Microsoft intern, or your thoughts or experiences.

Visualizing locations and paths on a map with Azure Maps

Mon, 23 Sep 2019 21:06:50 +0000

Azure Maps is a great way to add geospatial capabilities to your apps, from showing a map on a web page, to checking geofences, to routing and traffic data. You can read more about Azure Maps in the official documentation.

This post covers:

Showing an Azure Maps control on a web page
Adding pins to the map to show locations
Adding a path to the map

Getting started

Start by signing up for Azure if you don’t have an account:If you are a student, sign up at azure.microsoft.com/free/students to get US$100 of free credit and free services for a year.Otherwise sign up at azure.microsoft.com/free to get US$200 of credit for 30 days and a year of free services.
Sign into Azure and create an Azure Maps resource by following this link.

Once you have your Maps resource created, head to the Shared Key Authentication section of the _Settings -> Authentication _ tab in the map blade in the Azure portal and take a copy of your Primary Key.

Showing the Azure Maps control in a web page

Create a web site using your favorite tooling - I personally have been using Flask as I’m getting into Python. Create an HTML page. To the <head> section add the following to add support for the Azure Maps control:

<link rel="stylesheet" href="https://atlas.microsoft.com/sdk/javascript/mapcontrol/2/atlas.min.css" type="text/css">
<script src="https://atlas.microsoft.com/sdk/javascript/mapcontrol/2/atlas.min.js"></script>

In the <body>, add a div with an id of myMap.

<div id="myMap"></div>

Follow this with a script tag that creates the Azure Maps control and assigns it to this div:

<script type="text/javascript">
 var map = new atlas.Map('myMap', {
 language: 'en-US',
 authOptions: {
 authType: 'subscriptionKey',
 subscriptionKey: '{subscription-key}'
 }
 });
</script>

You will need to replace {subscription-key} with your primary key that you captured from from the Azure Maps resource blade.

If you view this page you will see the map control over the top part of the page. To size it, you can style the map to be the size of the whole page by adding a style to the head:

<style>
 html, body {
 margin: 0;
 }

 #myMap {
 height: 100vh;
 width: 100vw;
 }
</style>

This will give a nice, full size map showing the whole world.

You can read more about getting started with the Map control in the Getting started docs.

Zooming into a certain area

You can set the area that the map shows by setting the camera using a set of bounds and some padding - essentially giving the coordinates that must be included in the visible map and how much space you want around them. The map control works out the actual position and zoom to ensure the bounds and padding are visible, with extra map showing to ensure it fits on the page.

First your page will need to wait till the map control is ready before it can call any methods. You can do this using the ready event:

map.events.add('ready', function () {
 // add code to be run when the map is ready here
}

Inside this handler, you can set the camera using the setCamera method:

map.setCamera({
 bounds: [{min_long}, {min_lat}, {max_long}, {max_lat}],
 padding: 50
});

Set min_long and min_lat to be one corner of the bounding box that you want to ensure is visible using longitude and latitude coordinates. max_long and max_lat are the opposite corner. Set padding to the minimum number of pixels to have around the edge of the bounds.

For example, this code shows the area around Seattle:

map.events.add('ready', function () {
 map.setCamera({
 bounds: [-123, 47, -122, 48],
 padding: 50
 });
}

You can read more about this method in the Map class docs.

Adding pins for a location

Maps can have added layers such as pins and paths, with each layer showing coordinate data. This data is put into a DataSource, which gets added to the map, then a layer is applied to this data source.

To show pins on a map, the coordinates for the pins are added as a map feature to the data source, and a SymbolLayer is added referencing this data source. The map will show a pin for each point in the data source.

First, create the data source in the maps ready event handler:

var dataSource = new atlas.source.DataSource();
map.sources.add(dataSource);

Then create a data feature to wrap a point and add it to the data source. Features can include single points, polygons or lines, in this case it is a single point.

dataSource.add(
 new atlas.data.Feature(
 new atlas.data.Point([{longitude}, {latitude}])));

In this code, {longitude} needs to be set to the longitude of the point, and {latitude} set to the latitude. Finally a symbol layer is added to show the pins, and this symbol layer wraps the data source:

map.layers.add(new atlas.layer.SymbolLayer(dataSource, null));

There’s loads more you can do with these symbol layers, including customizing the symbols using images. You can read more and try some of these out in the Azure Maps Symbol Layer docs.

Showing a path on the map

As well as points, maps can also show a line layer. This is a series of points in a connected line, connected in the order the points are declared. This can be used to show a journey for example, tracking the route taken over time. You start with a data source, but this time with a flag set to show it contains line information:

var line_datasource = new atlas.source.DataSource(null, {
 lineMetrics: true
});
map.sources.add(line_datasource);

Next, a LineString feature is added to the data source:

line_datasource.add(new atlas.data.Feature(new atlas.data.LineString([
 [{point1_longitude}, {point1_latitude}],
 [{point2_longitude}, {point2_latitude}],
])));

Each point is added to the array passed into the LineString, in the order that you want the points to be connected. Finally a line layer is added to the map:

map.layers.add(new atlas.layer.LineLayer(line_datasource, null));

You can add multiple data sources to the map, and use them for multiple layers. For example you can have a data source for a line, and another for a point at the end of the line.

You can customize the line, such as the stroke size or color, including having color gradients to indicate the start and end of the line, by passing options when creating the layer. For example, to have a blue to red gradient, with red showing the end of the line (the last coordinate in the array passed to the data source), use the following:

map.layers.add(new atlas.layer.LineLayer(line_datasource, null, {
 strokeWidth: 6,
 strokeGradient: [
 'interpolate',
 ['linear'],
 ['line-progress'],
 0, "blue",
 1, "red"
 ]
}));

Summary

In this post we learned how to add an Azure Maps control to a web page, position the camera, add pins and add lines. Head to the official documentation to learn more about the capabilities of Azure Maps.

Are you where you should be? Checking Geofences using Azure Maps

Tue, 10 Sep 2019 00:08:58 +0000

A Geofence is a virtual boundary defined using an area on a map. Azure Maps has tools for checking if a coordinate is inside that Geofence. This post looks at how to do these checks.

There are many use cases for tracking where an item is, and if it moves into or out of a defined location. One great example is legal compliance - for example has an asset crossed an international or state boundary, is a vehicle on roads that require road tax to be paid (for example New Zealand’s RUC) or is a ship near a port.

One use case I’ve been thinking about recently is wildlife tracking. I’m building a sample app that can be used to track animal sightings including a GPS device that could be attached to an animal to provide 24/7 tracking (I’m not really going to put it on a real animal, this is just for demo purposes, so will be using a plushie bear). As well as knowing where an animal is, it would also be good to have an alert if the animal enters certain areas, for example if a bear goes near a center of population. This post shows how to create and check geofences using Azure Maps.

Getting started

Start by signing up for Azure if you don’t have an account:If you are a student, sign up at azure.microsoft.com/free/students to get US$100 of free credit and free services for a year.Otherwise sign up at azure.microsoft.com/free to get US$200 of credit for 30 days and a year of free services.
Sign into Azure and create an Azure Maps resource by following this link.

Defining a Geofence

Once you have an Azure Maps resource you need to define a geofence. These are defined using GeoJSON - a JSON document designed for geographic information. TO create a simple geofence for the Seattle/Redmond area, you would define it like this:

{
 "type": "Feature",
 "properties": {
 "geometryId": "1"
 },
 "geometry": {
 "type": "Polygon",
 "coordinates": [
 [
 [
 -122.41864, 47.54548
 ],
 [
 -122.41864, 47.70502
 ],
 [
 -122.00867, 47.70502
 ],
 [
 -122.00867, 47.54548
 ],
 [
 -122.41864, 47.54548
 ]
 ]
 ]
 }
}

This defines a geometric feature, that is a polygon using the coordinates given, with the last coordinate being the same as the first to close the shape - in this case a rectangle.

You can do a lot with GeoJson, and you can read more in the GeoJSON docs.

To set the geofence, you need to send this JSON to a call to the Azure Maps REST API. There aren’t any SDKs available for this service yet, but hopefully should be some soon.

Uploading a geofence needs a couple of calls - you make one call to upload which returns an endpoint that stores the data, then you call that endpoint to get back an Id you use to access the geofence from later calls.

The first REST call is:

https://atlas.microsoft.com/mapData/upload?
 subscription-key={subscription-key}
 &api-version=1.0
 &dataFormat=geojson

You will need to replace {subscription-key} with your primary or secondary key from the Shared Key Authentication section of the Settings -> Authentication tab in the map blade in the Azure portal.

The body of the request will need to be set to application/json and contain the GeoJSON for your geofence.

When you call this end point you should get back a status of 202 - Accepted. In the headers will be a Location containing an end point to call to get the Id of the geofence.

To get the Id, call the Location endpoint, adding your subscription key as an additional parameter:

https://atlas.microsoft.com/mapData/{location}/status?api-version=1.0
 &subscription-key={subscription-key}

This will return a status of 201 - Created, with a body containing a JSON document with a single field - "udId", the unique Id of the geofence.

Testing if a coordinate is in the geofence

The purpose of a geofence is to know when something goes in or out of it. This is done by making a call to another REST API, giving it the udId of the geofence, and the latitude and longitude of the coordinate you want to check.

https://atlas.microsoft.com/spatial/geofence/json
 ?subscription-key={subscription-key}
 &api-version=1.0
 &udId={udId}
 &lat={latitude}
 &lon=-{longitude}
 &deviceId={device-id}

In the above call replace {subscription-key} with your Azure Maps shared access key, {udId} with the udId from the second REST call, {latitude} and {longitude} with the latitude and longitude of the coordinate you want to check. The final parameter you need to set is {device-id}, and this needs to be set to an Id for the device that the coordinates come from. This device id doesn’t seem to be used for anything, so can be set to whatever you want, but it must be set otherwise you get an error.

The result of this call is a JSON document containing details about the location of the coordinates relative to the geofence.

{
 "geometries": [
 {
 "deviceId": "device",
 "udId": "xxxxxxxxx",
 "geometryId": "1",
 "distance": -999.0,
 "nearestLat": 47.54548,
 "nearestLon": -122.2
 }
 ],
 "expiredGeofenceGeometryId": [],
 "invalidPeriodGeofenceGeometryId": []
}

This JSON document returns the device and udId values passed in, useful if you want to pass this on to some form of notification system. It also gives a distance and the nearest latitude and longitude.

One thing to be aware of is GPS is not always exact - although calculations can be exact, coordinates are not always totally accurate. GPS sensors are at best accurate to a few meters, so a device could be inside the geofence but detected outside. The distance value takes this into consideration:

Positive distance values are outside the geofence, negative are inside.
If the location is within a short distance (default of 50m and referred to as a search buffer) of the edge of the geofence, the distance will be the distance to the geofence in meters.
If the device is further away from the edge, the value will be 999 if outside, -999 inside.
The search buffer can be configured in the REST call by setting the searchBuffer parameter to a value in meters from 0 to 500. If this is not set, the default of 50m is used.

When triggering alerts based off being close to the edge of a geofence you should take other information into consideration - for example if the GSP sensor is on a road going device, is the nearest road inside the geofence?

The nearestLat and nearestLon values give the point on the geofence that is nearest to the device - useful for example if you are tracking animals in an enclosed space, this might be where there is hole in the fence!

Learn more

If you want to learn more, check out these links:

Streaming Serial data using an MXChip

Wed, 04 Sep 2019 21:47:05 +0000

The MXChip board has 2 serial ports - one using the USB which is great for debugging, and one you can use to stream data from third party modules. This article shows you how!

TL;DR - connect the TXD on the module to 1 on the MXChip, RXD to 2, then create a new Serial using UARTClass Serial1(UART_1); You can then call this in the same way as the Arduino Serial class.

I’ve spent a lot of time working with the Azure IoT Dev Kit, also known as the MXChip. This is a great prototyping board that works seamlessly with Azure IoT Hub and comes with a stack of sensors built in. But what happens if you want to attach another device to it?

I’ve recently started creating a demo app based around a wildlife tracker - both spotting with a phone and an actual GSP collar. Not serious, not planning on putting it on an actual animal, just for a nice end to end demo of a scenario. Honestly - this is the bear that I’ll be tracking.

Hardware

The hardware I’m using for this is a cheap GPS module (note - mine didn’t come with mounting pins or an antenna). It sends serial data over UART at 9600 baud, and this data uses NMEA sentences - ASCII codes that contain GPS information including position, speed, satellite information and other data.

I plugged my GPS sensor into my Kitronic Inventors Kit, a breakout board for the BBC micro:bit that uses the same finger connector as the MXChip, found out which pins do serial data and connected some wires.

TXD on the GPS connects to Pin 1 on the Kitronic Inventors kit - the RXD pin (transmission on the module connects to receive on the MXChip and vice versa), RXD connects to Pin 2, the TXD pin. Power and ground comes from the chip as well, but can also come over USB if needed.

If you want to connect to the MXChip directly using crocodile clips, TXD connects to the large connector labelled 1, RXD connects to the one labelled 2, GND connects to the connected labelled GND, and VCC connects to the connector labelled 3V.

I had to solder pins to my GPS module to get a good connection as breakout pins didn’t work very well and kept on losing data.

Software

Next then tried to find the docs on accessing serial data. This is where I hit a brick wall - nothing! The Arduino docs recommend using a library that is shipped with it called SoftwareSerial, but this is not available in the MXChip libraries. The only thing that does work is using the built in Serial API to send data over the USB port.

I dug further, and the specs confirmed that there are 2 serial ports on the MXChip, so how can I access one? Eventually I found this post by Rob Miles:

https://www.robmiles.com/journal/2018/11/18/using-the-second-serial-port-on-the-azure-iot-devkit

You can create a new instance of UARTClass, the class that handles serial communication and connect it to UART_1, the second serial port. Once this is created, you can call it using the same API as the standard Serial instance:

UARTClass Serial1(UART_1);

Serial1.begin(9600);

while (Serial1.available() > 0)
{
 int byte = Serial1.read();
 ...
}

Once done I was able to read serial data from the GPS sensor. I then sent it to TinyGPS++ which I was able to install from the Android Library manager in VSCode. This decoded the data into latitude and longitude that I can then send to IoT Hub!

Save your users from typing with AI 🤖

Sun, 21 Jul 2019 21:52:08 +0000

I’ve been playing a lot with the Azure Custom Vision Service - a great service to build image classification and object detection models with just a few images. To make it easier to test my models, I wanted to build a mobile app that connects to a custom vision project and uses that to classify or detect objects in images captured using the camera.

The app is pretty simple, it’s a Xamarin app that uses a camera plugin and the Custom Vision SDKs. There is a problem with it… Configuration.

To use a model you need four pieces of information:

Prediction Key - a unique API key for you to use to classify or detect objects
Endpoint - these services can be run in different Azure regions, so you need the endpoint to show which region it is in
Project Id - you can have multiple projects, and each one is identified by a GUID
Publish name - each model has multiple iterations, and you publish the ones you want to be made available with a name

To use my model, I need to configure my app with all for of these pieces of information. I didn’t want to hard code them - I don’t want to have to rebuild my app every time I change project or iteration. Ideally I should be able to set these at runtime and store them between sessions.

I started off with a simple settings screen to type these 4 values in. I made a LOT of mistakes doing this as the values are long and complex. I needed to make my app more usable and find a way to avoid errors…

The solution was to use OCR - optical character recognition, using AI to convert text in an image to a string value.

Microsoft has a text recognition service available as part of the Azure Cognitive Services. This service takes an image, then detects text in it and returns that text.

Getting started

To get started you need either an Azure account. You can sign up for free here if you don’t already have an account.

You can access a free 7 day guest account without signup from here.

Once you are signed up, head to the Azure Portal to create a computer vision resource.

When you create this resource, you select what pricing tier you want - and there is a nice free tier allowing up to 20 requests per minute which is perfect for my app. Once created, you will need to grab the Endpoint from the Overview tab in the portal, and one of the keys from the Keys tab.

Using the service from code

There are a whole stack of SDKs available for Azure, covering the most popular languages and technologies, as well as a REST API for languages that SDKs are not available for. My app is a Xamarin app, so I used the .NET SDKs available on NuGet.

Install-Package Microsoft.Azure.CognitiveServices.Vision.ComputerVision -Version 5.0.0

Once the package was added, I just needed to take a picture and send it for processing. Taking pictures with a Xamarin app is not that complicated thanks to the Media Plugin, and I used this to take an image and give me a stream.

Once I had the image, I sent it to the Read API. This API takes an image and returns a token immediately whilst processing the image offline. You then call another API to check the status, and once the status comes back as processed you can grab the text.

Create the computer vision client

Before you can process an image, you need an instance of the ComputerVisionClient.

var creds = new ApiKeyServiceClientCredentials(<Your Key>);
var computerVision = new ComputerVisionClient(creds)
{
 Endpoint = <Your Endpoint>
};

In the code above, <Your Key> would be replaced with your API key for the computer vision service, and <Your Endpoint> would be replaced with the services endpoint.

Start the read request

To start using the Read API, you send the image to call to one of the BatchRead functions. There are several of these, depending on if you want to process a file or a stream. In my case I’m using a stream.

var results = await computerVision.BatchReadFileInStreamAsync(photo.GetStream());
var loc = results.OperationLocation;
var operationId = loc.Substring(loc.Length - 36);

The call to BatchReadFileInStreamAsync returns an intermediate result, and this has an OperationLocation - a token for the read operation. This is a long URL with the token on the end, so we need to extract the last 36 characters to get the token.

The result is an intermediate result, you will need to wait for the processing to finish

Wait for the request to finish

The request will take some time to run, as in a few milliseconds as opposed to minutes. The result of the batch call contains the status, and if this status is not complete then it can be retrieved continuously until it is complete, waiting between retries.

while (result.Status == TextOperationStatusCodes.Running ||
 result.Status == TextOperationStatusCodes.NotStarted)
{
 await Task.Delay(100);
 result = await computerVision.GetReadOperationResultAsync(operationId);
}

This code loops whilst the status is NotStarted or Running, pausing for 100ms then requesting the new status. Ideally you would also limit the retries and check for a Failed status.

Get the text

Once the status is Succeeded, the results will be available. The results come back as a list of TextRecognitionResult, each one referring to an area of text in the image. Each TextRecognitionResult contains one or more lines of text in the Lines property, and each line of text is accompanied by extra details such as the bounding box of the text. The lines are from top to bottom, so the topmost line of text would be the first entry in the Lines property, the next line down the next entry and so on.For my purposes, the settings page on the custom vision service has the data I need in a vertical list with the title of the field above the value, such as for the Project Id shown below.

In this case if I photographed these values and detected the text I would have one line containing the text "Project Id" with the next line being the GUID for the project Id.

I can now extract this value and use it in my app.

Done

Using the text recognition service from the Azure Cognitive Services means you will never have to ask your users to type in complicated data ever again!

You can find the code for my app on GitHub here: github.com/jimbobbennett/ObjectDetector. All the code that uses the cognitive services is in the SettingsViewModel.

Building a live caption tool - part 1

Tue, 02 Jul 2019 11:39:24 +0000

I’ve started a Twitch stream where I’m learning Python every Wednesday at 12pm UK time. One way I’d like to make my stream more accessible is by having live captions whilst I’m speaking.

What I need is a tool that will stream captions to something I can add to my OBS scenes, but also be customizable. A lot of off the shelf speech to text models are great, but I need something I can tune to my voice and accent, as well as any special words I am using such as technical tools and terms.

The Azure Cognitive Services have such a tool - as well as using a standard speech to text model, you can customize the model for your voice, accent, background noise and special words.

In this part, I’ll show how to get started building a live captioner in Python. In the next part, I’ll show how to customize the output.

Create the speech resource

To get started, you first need to create a Speech resource in Azure. You can do it from the Azure Portal by following this link. There is a free tier which I’m using - after all we all love free stuff!

If you don’t have an Azure account you can create a free account at azure.microsoft.com/free and get $200 of free credit for the first 30 days and a host of services free for a year. Students and academic faculty can sign up at azure.microsoft.com/free/students and get $100 that lasts a year as well as 12 months of free services, and this can be renewed every year that you are a student.

When the resource is created, note down the first part of the endpoint from the Overview tab. The endpoint will be something like https://uksouth.api.cognitive.microsoft.com/sts/v1.0/issuetoken, and the bit you want is the part before api.microsoft.com, so in my case uksouth. This will be the name of the region you created your resource in. You all also need to grab a key from the Keys tab.

Once you have your Speech resource the next step is to use it to create captions.

Create a captioner

Seeing as my stream is all about learning Python, I thought it would be fun to build the captioner in Python. All the Microsoft Cognitive Services have Python APIs which makes them easy to use.

I launched VS Code (which has excellent Python support thanks to the Python extension), and created a new Python project. The Speech SDK is available via pip, so I installed via the Terminal it using:

pip install azure-cognitiveservices-speech

To recognize speech you need to create a speechRecognizer, telling it the details of your resource via a speechConfig.

import azure.cognitiveservices.speech as speechsdk

speech_config = speechsdk.SpeechConfig(subscription=speech_key, region=service_region)
speech_recognizer = speechsdk.SpeechRecognizer(speech_config=speech_config)

In the code above, replace speech_key with the key from the Speech resource, and replace service_region with the region name.

This will create a speech recognizer using the default microphone. If you want to change the microphone you will need to know the device id and use this to create an AudioConfig object which is used to create the recognizer. You can read more about this in the docs.

The speech recognizer can be run as a one off and listen for a single block of speech until a break is found, or it can run continuously providing a constant stream of text via events. To detect continuously, an event needs to be wired up to collect the text.

def recognizing(args):
 # Do something

speech_recognizer.recognizing.connect(recognizing)
speech_recognizer.start_continuous_recognition()

In the above code, the recognizing event is fired every time some text is recognized. This event is fired multiple times for the same set of words, building up the text over time as the model refines the output. After a break it will reset and send new text.

The args parameter is a SpeechRecognitionEventArgs instance with a property called result that contains the result of the recognition. This result has a property called text with the recognized text.

For example, if you run this and say “Hello and welcome to the speech captioner”, this event will be called probably 7 times:

hello
hello and
hello and welcome
hello and welcome to
hello and welcome to the
hello and welcome to the speech
hello and welcome to the speech captioner

If you then pause and say “This works” it will be called 2 more times, with just the new words.

this
this works

The text is refined as the words are analyzed, so the text can change over time. For example if you say “This is a live caption test”, you may get back:

this
this is
this is alive
this is a live caption
this is a live caption text

Notice in the third result there is the word “alive”, which gets split into “a live” as more context is understood by the model.

The model doesn’t understand sentences, and in reality humans rarely speak in coherent sentences with a structure that is easy for the model to break up, hence why you won’t see full stops or capital letters.

The start_continuous_recognition call will run the recognition in the background, so the app will need a way to keep running, such as a looping sleep or an app loop using a GUI framework like Tkinter.

I’ve created a GUI app using Tkinter using this code. My app will put a semi-opaque window at the bottom of the screen that has a live stream of the captions in a label. The label is updated with the text from the recognizing event, so will be updated as I speak, then cleared down after each block of text ends and a new one begins.

You can find it on GitHub, to use it add your key and region to the config.py file, install the pip packages from the requirements.txt file and run captioner.py through Python.

In the next part, I’ll show how to customize the model to my voice and terms I use.

Adding Try .NET to VSCode launch.json

Tue, 11 Jun 2019 10:31:32 +0000

I’ve been playing a lot with Try .NET. I even blogged about it recently - [jimbobbennett.io/trying-out-try-net](/blogs/trying-out-try-net/).

One thing that was beginning to annoy me slightly was having to constantly launch the terminal and type dotnet try to test out what I was working in. My life would be infinitely improved (not really), if I could run it via F5 or the debug menu/tab instead of the terminal.

Turns out its pretty easy to do - just add a new entry to your launch.json file either directly from the file in the .vscode folder, or adding a configuration using the debug menu.

Add this to it:

{
 "name": "Try .NET",
 "type": "coreclr",
 "request": "launch",
 "program": "dotnet",
 "args":"try"
}

That’s all you need. Now you can run dotnet try just by pressing F5.

Trying out Try .NET

Tue, 04 Jun 2019 10:27:34 +0000

Try .NET is a new thing to come from the .NET teams that allows you to, well, try .NET. In a browser. You can think of it as a way to create interactive .NET documentation using .NET Core.

What is Try .NET

The experience or writing code in a browser itself isn’t new, developers have been able to try out C# coding snippets in the browser for a while but this is different. Try .NET is different, as it allows you to:

Mix code and markdown which then gets run as a web page where you can read the markdown and edit and run the code in place
Create projects with code, NuGet package dependencies, whatever is needed, then only surface the bits you want people to focus on

Note - Try .NET only supports C# at the moment, but F# is the most requested enhancement, obviously.

Mix code and markdown

Imagine you wanted to create a tutorial project to show how to do something in C#. In the past, you would write the instructions and provide code snippets for the reader to run by creating a project, then adding the code to a .cs file in that project, then running it from the command line.

With Try .NET, you can create a markdown file that links to a piece of code in an existing project. The reader can then run that code from inside the browser. They can also edit the code with full intellisense, get compiler errors if something isn’t write and generally play with the code.

Only surface what you need

When running code, you normally need some boiler plate stuff - a main method, some using directives, that sort of thing. If you want to run more advanced code that relies on external packages or setup code you need a way to bring those packages in or run the setup code. This can be complex, and fill up the code window with code that really isn’t relevant to what you want to teach.

Try .NET fixes this by having a C# project behind the scenes that you can set up how you want. You then write all the code you need, then surface just the bit you want to show in the markdown using #region directives - yup, finally a good reason for them.

Creating your first Try .NET project

Try .NET is a global .NET tool, and you can install it from a terminal or command line using:

dotnet tool install --global dotnet-try

It supports .NET Core 2.1, but for all the best stuff it’s worth installing 3.0.

You can spin up a demo project using:

dotnet try demo

but I thought it would be fun to walk through creating a project from scratch and go through how it all runs.

Create a dotnet project

Start with a simple dotnet console app:

dotnet new console -o TryDotNetDemo

In my case I’ve called my app TryDotNetDemo. Open this project in Visual Studio Code, then add a new file to the project called README.MD. This is a standard read me markdown file, the same as you would use in GitHub.

Add some markdown to this file to display a simple header and intro:

# A try .NET demo

This is a demo of Try .NET.

Run the code below to see some code being run.

From the Visual Studio Code terminal run the following:

dotnet try

Your code will be compiled, and launched in a browser where you will see your markdown rendered.

Add some code

Now we have some markdown, lets add some code. A simple Hello World will do for now.

Open the Program.cs file. The default main method will have a single line to write Hello World to the console.

using System;

namespace TryDotNetDemo
{
 class Program
 {
 static void Main(string[] args)
 {
 Console.WriteLine("Hello World!");
 }
 }
}

We want the Console.WriteLine call in the markdown, but not the rest of the code, so what do we do?

Well we add a region! Wrap the line in a named region block, using a name that makes sense.

# region HelloWorld
Console.WriteLine("Hello World!");
# endregion HelloWorld

Next, add an empty code block to your markdown:

```cs --region HelloWorld --source-file .\Program.cs --project .\TryDotNetDemo.csproj


Lets break this markdown block down:
```markdown
``` cs


These are the open and closing sections of the code block, shown by three back ticks. The closing set of three back ticks needs to be on a new line. The `cs` part tells the markdown renderer that the code in this block is C# code.
```markdown
--region HelloWorld

This tells Try .NET to look for a region called HelloWorld and put all the code inside this region into the rendered markdown.

--source-file .\Program.cs

This tells Try .NET to look for this region in the Program.cs code file.

--project .\TryDotNetDemo.csproj

This final option tells Try .NET that the code comes from the TryDotNetDemo.csproj file, so this project needs to be compiled and run.

Kill the running session in the Visual Studio Code terminal if it is still running, and relaunch it.

dotnet try

You will now see your code in the browser, and you can run it using the purple run button.

This is cool - but what’s even cooler is you can edit this code. Have a play - you get full intellisense and ctrl+space autocomplete, and if your code doesn’t compile you get proper compiler errors - with line numbers based off the code in the browser, not in the .cs file.

What’s happening

When your code runs, it will compile the console app with any code changes you’ve made included. If it fails to compile, you see the errors, otherwise the console app runs as normal - the main method is run and any output to the console is shown below your code.

Creating a more advanced project

This first example runs one bit of code, but what if you wanted to have multiple methods that can be edited, for example in a multi part or multi page tutorial? How would that work as the main method is run every time?

The answer comes in the form of command line parameters. When Try .NET runs your console app, it passes the region name, source file and project file as command line parameters to the app. To see this in action you can dump the parameters to the console and relaunch the code.-

Console.WriteLine(string.Join(' ', args));

You will see 6 parameters:

--region HelloWorld --source-file .\Program.cs --project .\TryDotNetDemo.csproj

These are the additional parameters added to the cs code block in the markdown.

You can use these to direct your code. A good pattern is to switch on the region in your main method, then call a different method.

static void Main(string[] args)
{
 switch (args[1])
 {
 case "HelloWorld":
 HelloWorld();
 break;
 case "Addition":
 Addition();
 break;
 }
}

static void HelloWorld()
{
 #region HelloWorld
 Console.WriteLine("Hello World!");
 #endregion HelloWorld
}

static void Addition()
{
 #region Addition
 int a = 1;
 int b = 2;
 int c = a + b;
 Console.WriteLine($"{a} + {b} = {c}");
 #endregion Addition
}

You can then surface both methods in the markdown:

# A try .NET demo

This is a demo of Try .NET.

Run the code below to see some code being run.

```cs --region HelloWorld --source-file .\Program.cs --project .\TryDotNetDemo.csproj

The code below shows addition:


When the `HelloWorld` region is run, the console app is called and `"HelloWorld"` is passed as the second argument to the app (the first argument is the `--region` parameter). The `switch` statement in the `main` method picks this up and runs the `HelloWorld` method only. Similarly the `Addition` region passes `"Addition"` to the app, and only the `Addition` method is run.
This allows you to break your app down into many methods that can be called in isolation inside your markdown.
## What about adding packages?
This is totally supported! Just add the package to your C# project as normal and use it in your code.
```cs
static void Json()
{
#region Json
var x = new { name = "Try .NET", status = "Awesome" };
var json = JsonConvert.SerializeObject(x);
Console.WriteLine(json);
#endregion Json
}

The code below shows some JSON handling with Newtonsoft JSON:

```cs --region Json --source-file .\Program.cs --project .\TryDotNetDemo.csproj


When you run this code in the browser it will use `Newtonsoft.Json.JsonConvert` to serialize an anonymous object to a string. You even get full intellisense for this package in the editor.


---
All the code for this is available on GitHub here - [github.com/jimbobbennett/TryDotNetDemo](https://github.com/jimbobbennett/TryDotNetDemo).
You can also read more on the [.NET blog](https://devblogs.microsoft.com/dotnet/creating-interactive-net-documentation/?WT.mc_id=trydotnet-blog-jabenn).

Connecting Pycom boards to Azure IoT Hub

Wed, 10 Apr 2019 12:38:16 +0000

I’m supporting an IoT hackathon over the weekend of April 27th/28th 2019. It will be a cool event, where you can win six months free business support from SETsquared – the world’s leading university business incubator. If you are interested, sign up here:

eventbrite.com/e/the-big-iothack-tickets-58111865153

As a part of this hackathon, PyCom are providing a load of devices. These are tiny IoT development boards that run MicroPython, and can connect over WiFi, BLE, LoRa or SigFox. PyCom have a service called PyBytes that can take data from the devices, and this can be integrated into Azure IoT Hub. To prepare for this event, I grabbed a board and started to play.

You will need an Azure subscription to work thorough this. If you don’t have one, you can sign up for free at azure.microsoft.com/free. This will give you $200 of credit, access to free services for 12 months, and other services free forever. You will need a credit card, but this is only for validation - you will NOT be billed unless to choose to upgrade your subscription.

Getting started

Unpack the board

We were given a big box of kit to try out before the hackathon. It had a range of boards and shields inside.

I grabbed a WiPy and a Pysense board.

The WiPy is a WiFi and BLE enabled board running an ESP32 micro controller. It doesn’t have any sensors or USB connectivity, it needs to be added to an expansion board.

The PySense is an expansion board that the WiPy plugs into. It has a load of sensors as well as a USB port you can use to program the micro controller.

Plug it in

The boards fit together, with the WiPy plugging into the socket on the Pysense. It took a bit of research to determine which way round to plug it in. Next I connected a WiFi antenna. Once it was all plugged together I connected the USB port to my Mac and an LED started flashing! Everyone loves blinking LEDs.

I then did the usual dance to upgrade the firmware by following the PyCom docs. As a heads up, if you are using a Mac, unplug everything from the USB ports first otherwise you’ll get errors as it tried to push the firmware to the wrong device!

Configure the software

Like a lot of developers, I love Visual Studio Code! PyCom provides an extension for Code that can talk to their boards.

Write some code!

These boards run MicroPython, an implementation of Python designed to run on micro controllers. MicroPython projects have the following structure:

Project folder
|-lib
| |- some_library.py
|- boot.py
|- main.py

Inside the project folder, there is an optional lib folder and two .py files. boot.py is optional and contains any code that you want run when the board boots up, such as connecting to WiFi. You can think of this as the same as the setup function in an Arduino C project. main.py is not optional, and contains the main code that will run on the device, analogous to the loop function in Arduino. It isn’t a loop as such, it won’t be called continuously, instead you will need to add your own processing loop. The lib folder is where you can put other .py files. This is the only directory that MicroPython will look in, you can’t put files in other directories or sub-directories of lib.

I created a new folder, and added main.py file. The code for this file is:

import pycom
import time

pycom.heartbeat(False) # Turn off the heartbeat

while True:
 pycom.rgbled(0xFF0000) # Red
 time.sleep(1) # Sleep for 1 second
 pycom.rgbled(0x00FF00) # Green
 time.sleep(1) # Sleep for 1 second
 pycom.rgbled(0x0000FF) # Blue
 time.sleep(1) # Sleep for 1 second

This code imports the pycom module containing code to interact with the board, and a time module to provide access to code to sleep.

The code starts by turning off the heartbeat - this is a regular pulse of the on-board LED that shows you the board is powered on.

Next it runs a loop - using a while True: loop to always run the code. The suite inside this loop sets the LED to red, sleeps for a second, then green, sleep, then blue, sleep.

Python is different to a lot of other languages in that sleep takes a time in seconds, not milliseconds.

To deploy the code, launch the command palette and select Pymakr->Run current file. This will compile the code and run it to the device. Once done, the LED will start to flash red, then green, then blue.

This only runs the code on the device, it doesn’t store it permanently, so once the device is rebooted, the code is wiped. To upload the code so that it is maintained between reboots, use Pymakr -> Upload project.

Send data to the cloud using PyBytes

Getting an LED flashing is cool, but what is cooler is getting data and sending it to the cloud. PyCom has a service called PyBytes that takes in data from PyCom devices.

Connect to PyBytes

PyBytes makes it easy to configure your device. You add a new device from their Web dashboard, set up a unique name for it and the WiFi details for the WiFi you want the PyCom device to connect to. You then get an activation token that you can use when flashing the firmware to push these details to the device.

Flash the firmware, and configure it for PyBytes, setting the activation token.

Once the firmware is updated, the device will appear in the PyBytes dashboard.

Set up the dashboard to receive some data

PyBytes manages data as signals. A signal is data sent from the device, identified by a unique number. The PyBytes dashboard takes the signal data, enriches it with a name and unit and allows it to be displayed on a dashboard.

From the Data tab on the dashboard, I defined a new signal called Temperature for ID 0, with the unit set to °C.

Send some data

Once the signal was created, I needed to send data to it from the PySense temperature sensor. There are libraries for accessing the PyBytes API, and the PySense board, and these need to be grabbed from the PyCom GitHub repo and dropped in the lib folder. I grabbed the [pysense.py](https://github.com/pycom/pycom-libraries/blob/master/pybytes/pysense/lib/pysense.py) and [pycoproc.py](https://github.com/pycom/pycom-libraries/blob/master/pybytes/pysense/lib/pycoproc.py) libraries to talk to the PySense board, and the [SI7006A20.py](https://github.com/pycom/pycom-libraries/blob/master/pybytes/pysense/lib/SI7006A20.py) library to access the temperature sensor.

The code I used to send the data is:

import pycom
import _thread
from pysense import Pysense
from SI7006A20 import SI7006A20
from time import sleep

pycom.heartbeat(False)

py = Pysense() # Connect to the PySense board
si = SI7006A20(py) # Connect to the temperature sensor

while True:
 pycom.rgbled(0x0000FF) # Flash the LED blue
 temp = si.temperature() # Get the temperature
 pybytes.send_signal(0, temp) # Send the temperature using signal 0
 pycom.rgbled(0x000000) # Turn off the LED
 sleep(5) # Sleep for 5 seconds

This uses the SI7006A20 temperature sensor to get the temperature, then sends a signal to PyBytes, using a signal id of 0. This data then appears on the PyBytes dashboard.

Integrate PyBytes with IoT Hub

PyBytes supports integrations with Azure IoT Hub, and it can push the data received from the device into IoT Hub.

Create an IoT Hub

I created a new IoT Hub instance to send the data to. I’ve got the Azure IoT Device Workbench extension installed in Visual Studio Code, and this provides capabilities to configure IoT services from inside Code.

From the command palette, selecting Azure IoT Hub -> Create IoT Hub allows you to create a new IoT Hub. Follow the instructions, and you get a new IoT Hub set up.

Integrate the IoT Hub into PyBytes

When the IoT Hub is running, open it up in the Azure Portal. Head to Settings -> Shared Access Policies, select the registryReadWrite policy and copy the Connection string - Primary key. PyBytes needs read and write permissions on the registry to register devices for you - each device configured in PyBytes becomes a device in IoT Hub.

From PyBytes, select Integrations -> New Integration. Select Microsoft Azure, then paste the connection string and select Login. Give the integration a name, enter a topic for the messages and select the devices you want to include. Then select Create. Once the integration is created, you can send a test message, then check for this in the IoT Hub.

You will also be able to see the device in the devices list both in the IoT Hub list, and in the Azure IoT Hub Devices section in the explorer. You can right-click on the device in the explorer and select Start monitoring D2C messages… to see a stream of the messages in the output window.

The data is raw, unenriched data showing the signal id and value.

Next steps

Now that the data is flowing into IoT Hub, you can do anything you want with it - enrich it using stream analytics or an Azure function, do time series insights on it, anything you want!

All the code for this project is available on GitHub here: github.com/jimbobbennett/PyCom-AzureIoTHub

Python on a Raspberry Pi using VS Code

Tue, 02 Apr 2019 14:32:05 +0000

I’ve recently been playing with a Raspberry Pi with the aim of learning Python. These are great little computers and have been an amazing success, selling 19 million as of March 2018, and kids everywhere are using them to learn to code with Scratch and Python.

The IDE for Python that is shipped by default is IDLE. This is a great IDE that comes as part of the standard Python install - it has an interactive shell and file editor, debugger and everything you need. It’s good enough, but not quite on par with the world class editor that is Visual Studio Code.

Currently there aren’t any official builds for VS Code for ARM processors, although there is a request for it on GitHub. Please head to this link and upvote it:

https://github.com/Microsoft/vscode/issues/6442

There is however a community build, courtesy of Jay Rodgers. You can find the installation instructions on code.headmelted.com, but I’ve found that these instructions don’t work due to an issue with the GPG key. There is a workaround!

To get the GPG key installed, launch a terminal and enter the following commands.

wget https://packagecloud.io/headmelted/codebuilds/gpgkey
sudo apt-key add gpgkey

Once done, you can run the installer script. Enter sudo mode using:

sudo -s

Then run the installer:

. <( wget -O - https://code.headmelted.com/installers/apt.sh )

This will install a non-VS branded version of code. The lack of Visual Studio branding is intentional - the code is open source, but the logos are copyright Microsoft. This means that a community build can’t use these icons. You can launch this from the menu, Raspberry->Programming->Code - OSS (Headmelted).

There is an issue with v1.32. It will launch, but not run properly. The fix is to rollback to an earlier version, v1.29. To do this, run this command:

apt-get install code-oss=1.29.0-1539702286

This rolls back to an earlier version. This does mean if you run apt-get update at a later date the latest version will be installed, breaking it again. To stop this happening, you can lock the version using:

apt-mark hold code-oss

Once a fix has been released, you can remove the version lock with:

apt-mark unhold code-oss

Once you launch code you will be able to install the extensions you need. In my case, I wanted the Python extension.

Quick restart later and I’m up and running with a full Python environment inside VS Code, with a full debugger and all the goodies I could ever want!

Jim made some dumb mistakes when recording a video. You won't believe how he fixed it with Azure Media Services!

Tue, 02 Apr 2019 09:38:50 +0000

I was at a recent Reading .NET Meetup, and wanted to record the event to make it available to members of the group who couldn’t attend the event in person. This is part of my aim to make meetups more accessible.

We have a cool box of recording git, courtesy of the Channel9 team at Microsoft, including a Canon XC10 camera. This camera records using the Canon MXF format, a propriety format that doesn’t work with my video editing app of choice, Camtasia. I also made a basic video recording error, I recorded the video at 30fps, and the audio using a separate recorder at 60fps!

This needed to be fixed - I needed to align the frame rates, and convert the MXF files to MP4 to work with Camtasia. And this is where Azure Media services comes in.

Azure Media Services is a suite of services that can encode and stream media. You create a media services resource, then you can use it to encode video either via the portal, or via the SDK.

Getting started

The first thing to do is to launch the Azure Portal and create a Media Service resource.

Once created, upload an asset - a video file to encode. From the Media Service resource, go to the Assets tab and select Upload. Point it to your .MXF file and it will start uploading. You can upload more than one file at once if needed.

Video files are huge - my smallest one was 6GB, so it takes a loooooooong time to upload if like me you have crappy bandwidth. Don’t let your computer go to sleep as this interrupts the upload and you have to delete the asset and start again.

Once uploading starts, you will be able to see the video file in the Assets tab. To encode it in the correct format, select the video file, then select encode.

From the Encode an asset tab, select the preset you want to use. The default is Content Adaptive Multiple Bitrate MP4, which provides adaptive streaming, changing the bitrate to match the streaming speed. This is great for content published on the net where viewers will be streaming it using different download speeds, but not so great for just re-encoding to bring into a video editing tool. Select an appropriate preset - I went for H264 Single Bitrate 1080p as I just want the highest bitrate and full size video. You can encode a video that is uploading - it will just queue the encoding job until the video is fully uploaded.

Again, this will take a long time!

Once the job is complete, head to the Assets tab and select the encoded asset. Before you can download the video file you need to publish the asset using a progressive encoder. Click the Publish button, set the encoder as Progressive and click Add.

The files for the asset wil be listed at the bottom, and one file will be a video/mp4 file. Select this file, and the details on it will expand out the side, including a download link. Use this link to download the encoded file.

Done! You can now import this file into tools like Camtasia.

Screen sharing a Raspberry Pi from a Mac

Sun, 31 Mar 2019 22:34:10 +0000

I’ve been playing with a Raspberry Pi for a while, and I’m getting fed up with changing the input to my monitor and using a second keyboard/mouse (yeah, yeah, 1st World Problem I know). I decided to set up a remote screen share so I can share the screen. This means I can use the same monitor/keyboard/mouse that I use for my Mac, but it also means I can un-cable myself and use my Pi from anywhere in the house, and even grab screenshots - useful for some upcoming blog posts I’m planning.

As it turns out, it’s really simple to get this set up.

Configure the Pi

Start by enabling VNC on the Pi. To do this, click the Raspberry Menu and select Raspberry Pi Configuration. From the configuration tool, select the Interfaces tab, then check the Enabled option next to VNC. Click OK, then restart the Pi when prompted.

Once the Pi reboots, you need to configure VNC. Click the VNC icon in the menu bar.

This will launch the VNC Server tool. Click the hamburger and select Options.

In the Security tab, set the Encryption to Prefer off and the Authentication to VNC Password.

In the Users & Permissions tab, select the Standard user (user), and click the Password… button. Set the password then click OK, then click OK.

Finally you need the IP address of the Raspberry Pi. Launch a terminal, and type ifconfig. Find the inet address from wlan0 and note this down.

Connecting from the Mac

To connect from the Mac, open Finder, then select Go -> Connect to server…. Enter the address as vnc://<ip address of the Pi>, for example vnc://192.168.2.10. Click Connect.

You will be prompted for a password. Enter the password you set up for your VNC standard user and check the box to remember this password. Click Sign In.

Screen sharing will launch, showing your Raspberry PI desktop!

Xamarin ❤ F#

Sat, 16 Feb 2019 20:35:51 +0000

Xamarin and Xamarin.Forms need no introduction. I’m sure by now you are well aware of these technologies and have seen many examples of building cross-platform mobile apps using C#. If not, I can highly recommend my book Xamarin in Action which if you follow this link and use code ‘xamarininaction’ you can get 40% off the cover price.

What you many not realize however, is you don’t have to just use C#. Yes, C# is a fantastic object-oriented language, but what if you like functional programming?

The answer is F#. F# is a functional-first programming language that is fully compatible with the .NET stack. What does ‘functional-first’ mean? Well it means it is a functional programming language that also supports object-orientation, classes and inheritance and whatnot. Essentially it encourages function programming whilst still supporting enough to interact with the OO world of the .NET framework.

For Xamarin developers, this is pretty awesome. Out the box F# is supported. This means you can build your apps in the same way as you would with C#, just using F#. You can write all your internal logic in a functional style, then take advantage of the OO features to interact with the APIs - derive a class from UIApplicationDelegate to configure your iOS app, derive a class from Activity for your Android app.

As is, this is pretty good combining the best of both worlds. This is far from perfect though for functional programmers - if you are going to use a functional programming language then you want to take advantage of everything the language has to offer, and not flip-flop from FP to OO style.

This dilemma has lead to a new framework being created, Fabulous. This framework sits on top of Xamarin.Forms and provides an MVU style architecture, putting a FP layer over the OO of UI code.

MVU stands for Model-View-Update and consists of 4 main parts:

Model - this is an immutable model containing all the state of your application. This model cannot change, and this is enforced by using an F# record type which is immutable by default. This allows you to stop worrying about the state of you application, as it is always in this model - in a single, well defined place.

Message - these define a start transformation request, and are triggered by either UI events like a button tap, or background events like a network call finishing or a push notification. These have a defined type, and can optionally have data taking advantage of the F# discriminated union type.

Update - this is a function that takes a model and a message and returns a new model. This is the only place that the state of the system can change, and is called synchronously removing any worries about race conditions.

View - this is a function that takes the model and returns a virtual UI based off the model. This virtual UI is lightweight, and can be thought of as analogous to HTML as text, a raw representation that is rendered on screen creating heavyweight UI components. The framework takes the virtual UI returned by this function and compares it to the actual UI. If any differences are found, the real UI is updated to match. This is called a differential update.

This design pattern brings the full power of functional programming to the OO based world of UI development, allowing you to craft great mobile apps using Xamarin, but sticking to the amazing power, flexibility and development pace of F#. And because it is built on Xamarin.Forms you get to maximize the amount of cross-platform code you write, but still can access all the native features you need, including being able to access the native controls being rendered, call any native API and even use native libraries from Java or Objective-C.

I recently spoke about this pattern at NDC London, and you can find my code, slides and some great links on my GitHub. The video of my talk will be up there soon.

Go check it out, I promise you once you start using Fabulous you won’t be disappointed.

Prototyping your first cloud-connected IoT project using an MXChip board and Azure IoT hub.

Wed, 06 Feb 2019 03:59:43 +0000

We all have cloud-connected devices. These are devices that do something taking advantage of sensors and hardware, then sync information via the cloud, such as sending sensor data or downloading configuration. For example, you might have a internet controlled thermostat in your home, that controls your heating and air conditioning, reports the temperature to the cloud so you can view it on your mobile device, and allows you to configure the temperature from your mobile device.

I decided to prototype my own cloud-connected IoT device by building an internet powered fan - a fan that comes on/goes off at a temperature threshold, reports the current temperature to the cloud, and allows you to control the on/off threshold also via the cloud. All this controlled by a mobile app.

You can find this project in GitHub here: https://github.com/jimbobbennett/InternetConnectedFan

This project contains all the code needed to build an internet connected fan using the MXChip Azure IoT Dev Kit and Azure IoT Hub.

This project detects the current temperature using the MSChips on-board temperature sensor, and if it is above a threshold, it will turn the fan on. It will also send telemetry data with the current temperature to an Azure IoT hub, which in turn triggers an Azure Function to save the temperature into a CosmosDB instance.

This project has a Xamarin mobile app that will poll another Azure Function to get the temperature and show it on screen. It also shows the threshold for turning on the fan, and this threshold can be configured from the app. Setting the value will call another Azure Function that updates the device twin for the device - a JSON document that is kept in sync between the device and the IoT hub. When the value on the hub is updated, it syncs to the device and the temperature threshold is updated, starting the fan if necessary.

Hardware

The hardware for this project is based around the MXChip which you can buy from here for around £40. This is an Arduino-compatible prototyping board that comes with a range of sensors, LEDs, buttons, a screen and other capabilities, as well as connectivity to Azure IoT Hub built in.

This then needs to be attached to a relay and a fan. For the fan, I used a simple battery powered fan that works with 2 AA batteries that is available here. I removed the batteries and connected wires instead, removing the on/off switch and running the cables through the hole. The fan runs on 3V so I was able to use power from the MXChip instead of batteries, as this board can supply 3V.

The relay I used was one that came with an Intel Edison kit, but any 3V relay will do. To make wiring easier, I used the Inventors Kit for the BBC Micro:bit - the MXChip has the same connector as the Micro:bit, and this kit converts the connector to a set of GPIO pins with a built in breadboard for easy prototyping.

Power to the relay poles was wired to the 3V/0V pins on the board, with PIN0 wired to the relay control. The input to the contacts was wired to 3V from the board, with the output connected to the +ve input on the fan. Finally the -ve input on the fan was wired to the 0V connection on the board.

Software

The software for this is in 3 parts - firmware for the device, some Azure Functions to work with device data and configure the device, and client code in the form of a mobile app.

Pre-requisites

To build and deploy the firmware and Azure Function app you will need:

A valid Azure account. Sign up for free here if you don’t already have an account.
VSCode
The VSCode IoT Workbench extension. This can be installed from inside VSCode.

To build and deploy the mobile app, you will need:

Visual Studio. This can be either Visual Studio 2017/2019 on Windows with the Xamarin workload installed, or Visual Studio for Mac.
To deploy on iOS you will need an Apple Developer Account.
To deploy to an Android device, you will need an Android device configured for developer mode.

Setting up the development environment

Start by opening the project.code-workspace file in VSCode.

Once you have the hardware assembled, you will need to configure the MXChip with a WiFi connection, create an Azure IoT hub and register the device. To do this, follow the instructions in the Getting Started Guide up to the Test the projectsection.

At this point, you will have an IoT hub configured, the device will be registered and and be configured with a connection string, and the code will be deployed to the device.

Firmware

The device code is in the Device folder in the project, and contains a single file called FanController.ino. This is an Arduino sketch file - a C file with the code for the firmware.

Arduino code has 2 functions called automatically - setup and loop.The setup function is called once when the device starts up, and is a good place to initialize the device, connect to WiFi etc. After that, the loop function is called repeatedly to do whatever your code needs.

The setup function for this project initializes the WiFi, connects to the IoT hub over MQTT including setting up a callback for updates to the device twin, then initializes one of the pins, PIN0, to send data. This pin is used to control the relay.

The loop function polls the sensor for the current temperature, writing it to the screen and sending it as a message to the IoT hub over MQTT.

If the temperature is above the threshold, a signal is sent to PIN0 to turn on the relay and send power to the fan, starting it up. The on-board LED is also turned to red.
If the temperature is below the threshold, the signal is stopped, turning the relay off and stopping power to the fan. The LED is set to blue. At the end of the function, it sleeps for 5 seconds to give a delay between polls of the sensor.

The device twin callback function, DeviceTwinCallback, is called whenever an update to the device twin is received by the board. This callback parses the JSON that is sent, extracting the threshold value.

When you worked through the setup, you would have deployed the code to the device, so this should be running already. If you want to re-deploy, use the Azure IoT Device Workbench: Upload Device Code option from the Command Palette.

When this app is running, you will be able to see telemetry on the messages received by navigating to your IoT hub in the Azure Portal.

Azure Function App

The Azure Function app lives in the Functions folder, and has 3 functions:

An event hub trigger that is called whenever a message is received by the IoT hub from the device. This saves the temperature to a CosmosDB instance.
A function to retrieve the temperature from CosmosDB
A function to update the temperature threshold using a device twin

This function app is written in C#, using Azunre Functions V2.

For the sake of simplicity, this app assumes you only have one device and the device id is fan-controller.

Before you can wire up the function app, you will need to create a CosmosDB instance using your preferred method (for example the Azure Portal). Ideally this should be in the same resource group as the IoT hub for easier resource management, such as deleting all the resources when you are done. Once this has been created, create a database called Devices containing a collection called Temperatures. Take a copy of one of the connection strings from the Read-write Keys tab of the Keys section as you will need this later.

To deploy the function app, use the Command Palette and select Azure IoT Device Workbench: Deploy to Azure…. Then select Function App. This will walk you through the steps to create a new function app in the same resource group as your IoT Hub, and deploy the code to it. Open the function app in the portal, and head to the Application Settings tab. Add 2 new settings:

CosmosDBConnection - set this to the CosmosDB connection string you copied earlier
ecs - set this to be the Event-hub compatible endpoint value from the Built-in endpoints section of the IoT hub.

After these settings have been updated, you should be able to use the storage explorer for the CosmosDB instance and see the temperature being written for the device. It will only write one record, overwriting it each time to save storage space.

You should now be able to retrieve the temperature and update the threshold using the REST endpoint, for example with a tool such as Postman. Make a GET request to https://<your function app>/api/temperature/fan-controller to get back a JSON document with the temperature for the fan-controller device.

{
 "partitionKey" : "temperature",
 "id" : "fan-controller",
 "temperature" : 24.6
}

To update the threshold, send a POST to https://<your function app>/api/temp-threshold/fan-controller with the following JSON body:

{
 "threshold" : <the_threshold>
}

setting <the_threshold> to be a double value of the threshold you want. For example, to set the threshold to 20°C use:

{
 "threshold" : 20.0
}

Mobile app

To build the mobile app, open the FanController.sln file in Visual Studio. Set the Android or iOS app to be the startup app depending on what device you want to run on.

You will need to update the urlRoot field in the FanController\MainViewModel.cs file to be the function app URL root - so <your function app>.azurewebsites.net without the https:// part.

Launch the app and you will see the temperature. Use the slider to change the threshold and tap the Set button to set this threshold. Once you tap Set you should see the threshold on the device change and the fan come on or turn off if applicable.

Flying long haul with young children

Tue, 22 Jan 2019 11:34:28 +0000

A discussion came up at work the other day on flying long-haul with very young children (toddler age), so I though it would be good to share the tips that I shared with the discussion. My daughter is 6 and has been on over 40 flights, including a lot of long haul as a toddler so I’ve managed to build up a selection of tips that have worked well for me.

Night flight. Long journeys are so much easier if they sleep lots. Try to plan how they will sleep seeing as they will be sitting up, so think about bringing cushions or pillows to make them comfortable. Remember - by default they don’t get their own seat so you will need to fit them on your lap.
Lots of spare clothes in hand luggage for both you and the child. Toilet accidents can happen, especially during turbulence if you can’t get them to a toilet. Also food spills happen, so you don’t want your little one in wet clothes for the entire flight if a juice box goes everywhere, nor do you want to sit there just in an airplane blanket because your trousers have 💩 on them. Pressure can affect bottles and juice boxes, so anything with liquid should be opened with care!
Something to suck on the way up and down – if they are still young enough to breastfeed then that’s perfect for up and down as it provides comfort even if they aren’t getting any milk, and helps with the pressure on the ears.
Bring lots of their favorite snacks and even some packed lunch style meals as airplane meals might not be to their liking, even a kids meal.
Eating with a child on your lap is hard - when the food comes round ask if one of you can eat first, then once they have finished the rest of the plane, bring food for the other one so one east whilst the other holds the child.
Get them used to headphones before you travel if they aren’t already. Bring those headphones with you, along with travel adapters for the dual plugs that some planes have.
Lots of bags with toys or sensory play so there’s something new every now and again. Child gets bored – open another.
Parents of young kids can board early, so take advantage of this. Also when leaving the plane wait for everyone else to get off. It’s just easier, especially to ensure you have rounded up everything. You don’t want to lose the favorite toy that your child can’t sleep without just because you were rushing off the plane with everyone else jostling for space. You have plenty of time to get off so it’s better to wait and double/triple check the important stuff is there. You’ve been on a plane for hours, so an extra 10 minutes doesn’t matter, especially as a lot of airpots will fast-track parents through immigration!
Take gifts such as pre-packaged chocolate for the crew because it’s a nice thing to do and can be a game for the child delivering presents to the cabin crew or ground staff. Happy cabin crew will be extra helpful when needed on the flight.
Have your hand luggage well organized so you know exactly where to go to get food/wipes/entertainment. Space is limited so you don’t want to have to fight through multiple bags to get something.
Spare batteries and power packs for any gadgets they have on the plane – if their iPad runs out 1 hour in it’ll be a long flight! Also make sure what you think you have downloaded is actually there on the device and works in airplane mode.
In the run up to the flight make it exciting. My daughter has an Usbourne flap book on airports and airplanes and loves it. She used to take it on all her flights and find the right page for the thing we were doing (check in, boarding, take off etc.).
Be selfless with your partner. It will be trying for you both so if it does turn stressful laugh about it instead of fighting. Take turns to walk away and hang out in the galley if you need a break. Miserable parents make for a miserable flight so focus on happiness - maybe even pack a few hidden notes to each other saying things like “Sorry I’m grumpy as I have poo on me, I still love you” to hand to your partner at the appropriate time, or a hidden stash of their favorite chocolate to cheer them up.
Ignore anyone around you who complains without reason. Young children are allowed on a plane, so if the grumpy old man behind you complains about noise or demands up front that your child is silent and doesn’t cry all flight (has happened to us), then ignore them and don’t get stressed. Be nice, be apologetic if needed, but be fair to yourselves. You all have a right to travel.

Hope this is helpful to travelers! Enjoy your flight and let me know of any other top tips so I can update this list.

Fabulous Santa Tracker

Mon, 24 Dec 2018 00:00:00 +0000

‘Twas the night before Christmas, and all through the house…

For millions of people worldwide, Christmas eve is a time of magic. Children are excited and unable to sleep because…

Santa is coming tonight!!!!

This is the one night of the year you want a stranger to come into your house whilst you sleep. He’ll magic his way in (cos who has a chimney these days, right?), and leave presents under your tree.

He know if you are sleeping, He knows if you’re awake…

But - and this is important, he only comes if you are asleep. So children, you need to be asleep early. But how early? How do you know when you need to be asleep by?

For this - you need a Santa tracker mobile app. And not just any Santa tracker, a Fabulous one!

This app is pretty simple. It knows where Santa is at any given time on Christmas eve, and based on the current time displays the location of Santa on a map, along with a running count of how many presents he’s delivered so far to all the good girls and boys.

The code is all on GitHub if you want to check it out.

Building the Santa tracker

To build this app I needed a source of data, and a mobile app.

Where did the data come from?

Every year for the past decade or so, NORAD has been tracking Santa online every Christmas eve. Unfortunately they don’t seem to have a public API that I could find to use. Luckily Google has got in on the action, and provides an undocumented API to download a JSON file containing Santas coordinates from last year. I simply downloaded that JSON document to use. The structure is pretty simple, and I’ve shown a snippet of it below showing only the bits I’m interested in.

{
 "destinations": [
 {
 "arrival": 31536000000,
 "presentsDelivered": 0,
 "city": "Santa's Village",
 "region": "North Pole",
 "location": {
 "lat": 84.6,
 "lng": 168
 }
 },
 {
 "arrival": 1514110140000,
 "presentsDelivered": 46415,
 "city": "Provideniya",
 "region": "Russia",
 "location": {
 "lat": 64.436249,
 "lng": -173.233337
 }
 ...
 ]
}

This gives a city and region name for each stop, total presents delivered, the latitude and longitude of the location and the arrival time. This arrival time is the number of milliseconds since the UNIX epoch - 1 Jan 1970.

Creating the mobile app

For the mobile app, I wanted to use F#, and there is a great F# mobile framework called Fabulous!

Fabulous?

If you haven’t heard of Fabulous yet, its an F#-based MVU framework (so like Elm) for building cross-platform mobile apps, built on top of the Xamarin.Forms platform. You can read all about it in the docs. There’s even an awesome-fabulous list containing cool projects that use it and some great resources.

You can create a new Fabulous app using the dotnet CLI. You install the template using:

dotnet new -i Fabulous.Templates

The create the app using:

dotnet new fabulous-app -n FabulousSantaTracker

This will create an F# solution with an Android app, an iOS app and a .NET standard library containing all the cross platform business logic and UI code.

Reading the JSON

Processing the JSON file is easy. First I added the downloaded JSON file to the core project, setting the build action to EmbeddedResource. Once I had this, I added a new F# source file and declared some new types to store the data.

type Location = { Lat : float; Lng : float }

type Destination =
 {
 Arrival : float
 PresentsDelivered : int64
 City : string
 Region : string
 Location : Location
 }

type Destinations = array<Destination>

type SantaData = { Destinations : Destinations }

To read the JSON data I needed to read the embedded resource, then use Newtonsoft.JSON to deserialize it (the NuGet for this is included in the Fabulous template, so nothing extra to install).

let GetResourceString fileName =
 let assembly = IntrospectionExtensions.GetTypeInfo(typedefof<Destination>).Assembly
 use stream = assembly.GetManifestResourceStream(fileName)
 use reader = new StreamReader (stream)
 reader.ReadToEnd()

let AllDestinations =
 let santaData = JsonConvert.DeserializeObject<SantaData>(GetResourceString "FabulousSantaTracker.santa_en.json")
 santaData.Destinations

The GetResourceString function loads the assembly and extracts the resource string from it. The name of these resource strings needs to be namespace qualified. For example the file in my project is called santa_en.json, and my namespace is FabulousSantaTracker, so the full resource name is FabulousSantaTracker.santa_en.json.

The last bit I needed was to convert the arrival date from milliseconds after epoch. These arrival dates are for last year (2017), so I needed to update them to the current year.

let convertFromEpoch ms =
 let d = epoch.AddMilliseconds (ms)
 d.AddYears(DateTime.UtcNow.Year - d.Year)

Now I have my data, it’s time to display this on a map. Fabulous uses an MVU architecture, so I need to define a model, some messages, an update function to process these messages, and a view function.

Defining the model

The model is simple - it just needs Santa’s current location by finding the latest destination in the list with an arrival time before now by working backwards through the list of destinations loaded from the JSON file. If Santa hasn’t arrived anywhere yet then the first destination is used - this is Santa’s workshop at the North Pole.

let currentDestination () =
 let current = TrackingData.AllDestinations |> Array.tryFindBack (fun i -> i.ArrivalDateTime < DateTime.UtcNow)
 match current with
 | Some d -> d
 | None -> TrackingData.AllDestinations |> Array.item 0

type Model =
 {
 CurrentDestination : Destination
 }

let init () = { CurrentDestination = currentDestination() }, Cmd.none

Updating the map when Santa moves

The easiest way to update the current destination when Santa moves is a simple timer - fire a timer every few seconds and update the current destination. This can be implemented by defining a message for a timer tick, and when this is handled in the update function, update the models CurrentDestination on every tick.

type Msg =
 | TimerTick

let update msg model =
 match msg with
 | TimerTick -> { model with CurrentDestination = currentDestination() }, Cmd.none

This message can then be fired using a subscription to a timer. These subscriptions are functions that take the Fabulous dispatcher and are called when setting up the program.

let timerTick dispatch =
 let timer = new Timer(TimeSpan.FromSeconds(10.).TotalMilliseconds)
 timer.Elapsed.Subscribe (fun _ -> dispatch TimerTick) |> ignore
 timer.Enabled <- true
 timer.Start()

let runner =
 App.program
 |> Program.withSubscription (fun _ -> Cmd.ofSub App.timerTick)

Once this subscription is called, the timer is started and every 10 seconds the TimerTick message is dispatched. This causes the models CurrentDestination to be re-evaluated and the view then gets updated.

Drawing the view

Drawing the view is pretty simple. Fabulous uses a virtual UI, so your view function always returns a complete UI, and the internals of Fabulous compare the current with the result of the view call and applies the deltas to the real UI.

My view starts with a navigation page containing a content page with some emoji in the title:

let view (model: Model) dispatch =
 View.NavigationPage(
 pages = [
 View.ContentPage(
 title = "🎅 Tracker"
 ...
 ]

It then contains a grid with 3 rows. First row for the current location, second for the number of presents, third is a map.

The label to show the location has the text set like this:

View.Label(text = model.CurrentDestination.City,

This is one of the ways MVU differs from MVVM (the canonical design pattern for Xamarin.Forms apps) - the value for the location isn’t bound to anything, it’s just set using the value from the model. Updates to this value can only come from the update function, and after updating, the view function is called and the label is created with the new text. The differential update in the Fabulous framework will see this value change and update the UI.

To show the position on a map, we can use the Fabulous.Maps NuGet package which contains a wrapper for the Xamarin.Forms Map element. These wrappers are the way that Fabulous can provide a virtual UI, so is needed for each component you want to use in your view function. Once this NuGet is added, the map can be added to the view function.

View.Map(
 hasZoomEnabled = true,
 hasScrollEnabled = true,
 requestedRegion = MapSpan.FromCenterAndRadius(model.CurrentDestination.Position, Distance.FromKilometers(1000.0)),
 pins = [
 View.Pin(
 model.CurrentDestination.Position,
 label = "Santa",
 pinType = PinType.Place
 )
 ]
)

This code creates a map, sets the region to the position of the current destination (the position is an instance of the Xamarin.Forms Position class using the latitude and longitude from the JSON file) and zooms out to show a radius of 1,000Km. It will also add a pin at the position so that the user can see the position more accurately.

Using a custom map pin

This code so far shows where Santa is, and updates the UI when he moves, but doesn’t look the best as the pin is, well just a map pin. It would be better to show Santa himself.

To do this I added some images to the iOS and Android apps of Santa, drawn by using the Santa emoji. To show these as a custom pin means I need to dive down into some platform specific code. One of the upsides of Xamarin.Forms, the underlying technology for Fabulous, is that you have access to the native UI components, so can update them as if your app was a fully native swift/java app.

Accessing this native code involves the use of custom renderers - custom code to help Fabulous renderer the native controls. Customizing the map means creating my own SantaMap class that derives from the Forms Map class.

type SantaMap() =
 inherit Xamarin.Forms.Maps.Map()

You can see the full implementation of this on GitHub. There is a lot of code here, but is lifted from the original maps implementation so was minimal work. Hopefully in the future this will be easier!

Once this was in place, I needed to create the custom renderers to draw different pins. The Android one is:

type SantaMapRenderer(context : Context) =
 inherit MapRenderer(context)

 override this.CreateMarker(pin : Pin) =
 (new MarkerOptions()).SetPosition(new LatLng(pin.Position.Latitude, pin.Position.Longitude))
 .SetTitle(pin.Label)
 .SetIcon(BitmapDescriptorFactory.FromResource(Resources.Drawable.Santa))

module Export_SantaMapRenderer =
 [<assembly: ExportRenderer(typeof<SantaMap>, typeof<SantaMapRenderer>) >]
 do ()

Pretty simple - it uses the existing renderer and overrides the code to create the pins to use my Santa image. These renderers need to be registered with the framework to be used, and this can be done using the ExportRenderer attributes.

Fin!

That’s all there is to this - a Santa tracker built using Fabulous. You can find all the code on GitHub here: https://github.com/jimbobbennett/FabulousSantaTracker

To build this app for iOS, just build it and run it and on Christmas eve watch for where Santa is. Android is a bit more effort as it needs a Google Maps key. You can find instructions on getting one in the Xamarin Docs. Once you have your key, add it to the AndroidManifest.xml file and track Santa!

Merry Christmas!

Open anything in VS Code using a MacOS quick action

Thu, 13 Dec 2018 14:29:25 +0000

I’ve been getting fed up with the long winded process to open things in VS Code - you know, typing in the terminal, opening VS Code and dragging folders in, that sort of long, manual process… #FirstWorldProblems 😜

I thought it would be fun to automate the whole thing, using Automator!

If you haven’t used Automator, it’s a great little tool. You can build simple workflows using capabilities from system apps and other apps, configure the data that comes in, then trigger these when things happen, such as calendar events, items changing in a folder or on demand.

These workflows can be wired up to “Quick Actions”, these are actions that can be applied to text or images and are called from the context menus in Finder or in a text box. They can also be launched from a keyboard combination or the touch bar.

Creating a Quick Action to open VS Code

The first step is to create a quick action to open VS Code.

Launch Automator
Create a new Quick Action

Configure the workflow:
- Set it to receive current “files and folders” from Finder.
- Set and image if you want one - I used the icons from this GitHub repo.
Add a new Run Shell Script action to the workflow.
- Set the Pass Input to be “as arguments”
- Set the shell script to be:

open -n -b "com.microsoft.VSCode" --args "$*"

Save the action using a name like Open In VS Code.

This will register the quick action with Finder, and you will now be able to right-click on a file or folder in Finder and select Quick Actions->Open In VS Code

Adding a keyboard shortcut

Keyboard shortcuts can be added to Quick Actions from System Preferences.

Launch System Preferences
Open the Keyboard settings and select the Shortcuts tab
Select Services from the list on the left
Scroll down on the right to Files and Folders->Open In VS Code
Click the none keyboard shortcut and press the keys you want to use

Adding to the touch bar

If you are a fan of the touch bar (and I am, and I know it’a a controversial opinion), you can add quick actions there. Unfortunately not on an action-by-action basis, but you can add a button that shows the quick actions.

From the Keyboard settings in System Preferences, select the Keyboard tab.
Click Customize touch bar…
Drag the Quick Actions button down to the touch bar and position it where you want. For mine I replaced the default Siri button with the quick actions button as I’ve never used Siri on my Mac.
Click Done

You will now have a quick actions button on the touch bar, and when you tap it it will show you the Open in VS Code action. If you change the color of the quick action in Automator it will change the color of this button!

More shortcuts…

Once I started with this, I added one to open a folder in terminal, and will probably add a load more for things like git commands.

Controlling an IoT device from your phone

Mon, 19 Nov 2018 17:26:51 +0000

I’ve just finished watching the latest episode of Five Things, where Suz Hinton talks with Burke about Goo Goo Clusters and IoT devices.

One of the devices Suz showed was the Azure IoT DevKit - an arduino compatible board made by MXChip that works beautifully with Azure, even down to having an Azure LED on board to show when it is connected. This is a great little board with a whole load of sensors and other gubbins built in.

I have one of these boards thanks to Suz, so I thought it would be a good opportunity to get it out again and see how easy it would be to get my phone to control the board via Azure, with the eventual goal of doing some kind of holiday hack such as using it to control Christmas lights.

Getting started with the IoT DevKit

Buy one! Lots of links here - http://mxchip.com/az3166. These boards are around $40 so not too expensive - maybe put one on your Christmas list.
Follow the first part of the Getting Started guide to prepare your hardware, configure WiFi and install the development environment.
Sign up for an Azure account. If you don’t already have an Azure account, you will need to sign up for one. There is a free tier that at the time of writing gives US$200 for the first 30 days, loads of services for free for 12 months, and a range of services free forever. You can sign up for the free tier here!

Once done your board will be ready to program.

Controlling the board from the cloud

My plan is to build an app where I can set a color and have the LED on the board match that color. The architecture I want is something like this:

Part of this is easy - a mobile app talking to an Azure Functions back end over a REST API. The new bit for me is the big ?, the bit where the IoT device talks to the back end.

Azure IoT

Azure has some great services for managing IoT devices, all based around IoT Hub. IoT Hub is a managed service that provides a central resource for two-way communication with IoT devices. You can register devices in the hub, configure them individually or en-masse, send and receive data using standard messaging protocols like MQTT, or manage the devices including updating firmware.

One feature that it also supports is device twins.

Device twins are JSON documents that store information such as state or configuration. These exist on both the device and the Azure IoT hub and are kept in sync, so if you update a value on the device it syncs to the cloud and vice versa. This is perfect for my app - I can encode the RGB value for the LED into a device twin and update it from my Azure function. This update will then be pushed to the device where it can be parsed and the LED color changed to match.

If you want to see this in action, then there is a great demo project available here that creates a web site that can turn the LED on/off and set the color. It doesn’t go into how it works, just shows you that it does so I thought it would be more fun to try to build it myself. The rest of this post covers how I did it instead of just following this walkthrough.

Creating an IoT hub

The first thing I needed to do was to create an IoT workbench project and set up the IoT hub and device. This can be done from inside VSCode using the Azure IoT Workbench extension. Launch VSCode, then plug in the IoT Dev Kit.

To create a new IoT workbench project:

Launch the command palette and select IoT Workbench: New.
Give it a folder to create the project in
Give the project a name
Select the IoT DevKit device.
Create a project with an Azure IoT Hub
Use the default sketch file name (sketch files are C code files that are run on the IoT device)

Once the project has been created, the IoT hub needs to be created and the device needs to be registered with the hub. Again, this can be done from inside VS Code.

Launch the command palette and select IoT Workbench: Cloud
Select Azure Provision
Select a subscription, signing into Azure if necessary
Create a new resource group or select an existing one
Create a new IoT Hub in a location near you
Select the F1 free tier. If you already have a free tier set up for another project then use an S tier - the B tiers don’t support device twins, and the F1 tier acts like an S tier but with limits on messages.
Give the IoT Hub a name - this will need to be globally unique across Azure.
The IoT Hub will now be provisioned, and this will take a few minutes to complete.

Once the Hub has been provisioned, the device will need to be provisioned, and this step will start automatically once the Hub has been created.

Create a new IoT device
Give the device a name. Each device will need to have a unique name in the IoT hub.

Once done you will be able to navigate to your IoT Hub in the Azure Portal and see your device.

Finally the IoT device needs to know the connection string so that it can connect to the IoT Hub.

Put the IoT device into configuration mode by holding the A button on the device, tapping the Reset button, then releasing the A button.

From the command palette select IoT Workbench: Device
Select Config Device Settings
Select Config Device Connection String
Select Select IoT Hub Device Connection String

A notification will pop up once done. The IoT device can then be rebooted by disconnecting and re-connecting it, and when it’s booted it will be connected to the IoT Hub, as indicated by the blue Azure LED being lit.

Adding code to handle the device twin

The device is configured, so now it needs some code to detect updates to the device twin and update the LED color.

The code for this is here. Copy this code into the sketch file in the IoT project (the .ino file). The interesting parts of this code are the setup and loop methods.

The setup method

void setup()
{
 ...
 DevKitMQTTClient_Init(true);
 DevKitMQTTClient_SetDeviceTwinCallback(DeviceTwinCallback);
}

The setup method is a special method that is called every time the device starts up or is rest. This method is used to initialize things, such as the WiFi and connectivity to IoT Hub.

The DevKitMQTTClient_Init call initializes the MQTT client to connect to Azure IoT Hub.
The call to DevKitMQTTClient_SetDeviceTwinCallback sets up a callback for any updates to the device twin on the IoT Hub. In the DeviceTwinCallback function, the JSON payload is parsed looking for one of these values:
- "rgbLEDR" - The red value for the LED from 0 to 255
- "rgbLEDG" - The Green value for the LED from 0 to 255
- "rgbLEDB" - The Blue value for the LED from 0 to 255
Once these values are found they are stored in global variables.

The loop method

The other method of interest is the loop method. This method is a special one that is called repeatedly by the OS on a loop. This method sets up the LED based on the on/off and RGB values stored in the global variables then applies these to the LED. It then sleeps for a bit and loops again.

Deploying the code

Once you’ve added the code, it needs to be deployed to the device:

Select IoT Workbench: Device from the command palette
Select Device Upload

The code will be compiled, uploaded to the device, and the device will reboot. After rebooting it will be ready to sync data using the device twin.

Once the code has been deployed, the IoT device can be unplugged from your computer and connected to another power supply if needed. The connection to a computer is only needed for configuration and uploading code, when running it just needs power through the USB port, nothing else.

Testing out the device twin

Once the device reboots, it will start listening to updates to the device twin. The device twin can be updated in code, but a simple way to get started is using the Azure portal.

From inside the portal:

Open up the newly created IoT hub
Select Explorers->IoT devices
Select the IoT device from the list
Select Device twin from the menu across the top

This will show the device twin JSON document in an editor. This document can be updated, and once the update is saved it will be sent to the device. To turn the LED on on the IoT device, values for rgbLEDR, rgbLEDG and rgbLEDB need to be added to the existing properties->desired node in the JSON document:

{
 ...
 "properties": {
 "desired": {
 "rgbLEDR": 0,
 "rgbLEDG": 255,
 "rgbLEDB": 0,
 ...
}

Once these values are added and the document saved, the LED will light up in bright green. Then changing these values will cause the LED to change color.

Updating the device twins from an Azure function

The device is working, so the next step is to create a back end that can be called from a mobile app to update the device twin. For this I’m going to use an Azure Function. This will be an HTTP trigger that listens for a POST with a body containing JSON with values for red, green and blue. This JSON will be parsed and the values used to update the digital twin:

{
 "red" : 0,
 "green" : 0,
 "blue" : 255
}

To help with Azure IoT projects, there is a Microsoft.Azure.Devices NuGet package. This contains a RegistryManager class that can be used to manage devices, and this RegistryManager is created using a connection string for the IoT Hub.

Each IoT Hub can have multiple connection strings, all based around their different permissions. To control a device twin you need these permissions:

Registry Read - Allows reading from the IoT registry
Service Connect - Allows connections to the IoT Hub to manage devices and setting, this is designed for services to connect, as opposed to devices connecting.

From inside the portal:

Open up the newly created IoT hub
Select Settings->Shared access policies
Click + Add to create a new shared access policy, call it Twin and allow Registry Read and Service Connect
Save this, and copy one of the connection strings using the Copy button next to it

Once the connection string has been created it can be used to create the RegistryManager, ideally by setting this in the function app settings.

var connectionString = Environment.GetEnvironmentVariable("iotHubConnectionString");
var registryManager = RegistryManager.CreateFromConnectionString(connectionString);

The device twin can then be retrieved using the RegistryManager and the name of the IoT device that was used when it was registered in the Hub (again ideally as a setting):

var deviceName = Environment.GetEnvironmentVariable("iotHubDevice");
var twin = await registryManager.GetTwinAsync(deviceName);

This twin can then be updated by passing a JSON string containing a patch to apply, and the values in the patch are applied to the twin. In this case, the rgbLEDR, rgbLEDG and rgbLEDB values need to be added to the desired node in the properties node, and this can be done by creating an anonymous type with the right names and serializing it to JSON.

var patch = new
{
 properties = new
 {
 desired = new
 {
 rgbLEDR = red,
 rgbLEDG = green,
 rgbLEDB = blue,
 }
 }
};

var jsonPatch = JsonConvert.SerializeObject(patch);

This patch is then used to update the twin:

await registryManager.UpdateTwinAsync(twin.DeviceId, jsonPatch, twin.ETag);

That’s all that is needed to update the twin. The full code for this function is available here. To test this out:

Set the appropriate values for the connection string and device hub in the local.settings.json file
Run the function app using the local functions host
Post a JSON document to the function using a tool such as Postman
The LED will change color based on the JSON that was posted

Once the function is running, it can be deployed to Azure so that it can be called from a mobile app. Once deployed, the application settings will need to be set for the IoT Hub connection string and device name. This function can then be called by a tool like Postman to verify it is working.

Building the mobile app

The mobile app is pretty simple - three sliders and a button. Use the sliders to set RGB values, the button changes color as they slide, and when you tap the button it sends the RGB value to the back end. I built the app using Fabulous, a F# MVU architecture that sits on top of Xamarin. The code for this is pretty simple, with the values from the sliders being passed to an HTTP.AsyncRequest from FSharp.Data. This calls the Azure Functions back end to update the device twin.

Here’s the app running in all it’s glory!

Summary

This post showed how to control the LED on an MXChip board using Azure device twins.

All the code for this is on my GitHub.

Here’s some further reading:

Learn more about the Azure IoT hub here
Places to buy an MXChip board
Microsoft.Azure.Devices API docs: docs.microsoft.com
Fabulous getting started guide
Goo Goo Clusters - feel free to buy me some!

Azure makers series

Wed, 14 Nov 2018 17:09:50 +0000

I recently recorded a quick video to talk about an app I was building using the Azure free account. You can check it out here:

If you want to see the code, it’s in my GitHub repo here - https://github.com/jimbobbennett/AzurePhotoSharer

Trust

Tue, 13 Nov 2018 18:11:34 +0000

Thought I’d repost a blog post I wrote 4 1/2 years ago on Medium onto this blog - be interested in what readers think.

Shouldn’t our default position be to trust first?

I was having a conversation with a friend of mine recently about his current work situation. His team leader has a default state of not trusting in the knowledge and abilities of the team. Systems are locked down so only the team leader can access them, decisions are made by said leader and bypass the knowledge, skills and abilities of the team. This was raised up the chain by one of his co workers only to be told to spend time earning that trust (despite already being there for a while). This has made for an unhappy team with one already leaving and a few more making plans in that direction.

This is a very different model to how I like to run teams. I want to have a team working with me where I can trust in the skills of every member — instead of locking them out I open everything up to them. If they lack the right skills, instead of blocking them from doing a task I’d rather work with them to help them learn how to do it. Put them in a place where they can learn through self discovery, learn through failure with a guiding hand to help them succeed next time. I find this makes for a very cohesive team, everyone feels valued, they matter to the success of the team and they feel their skills growing. To quote Peopleware by Tom DeMarco and Timothy Lister “Once you have decided to go with a certain group, your best tactic is to trust them … People who feel untrusted have little inclination to bond together into a cooperative team”

I feel very comfortable with my model. I always try my hardest to ensure that I interview thoroughly so that I know what my new hires can and can’t do. I also spend time with the team on a one to one basis to constantly get and give feedback on how they are doing as a team member and I am doing as their team lead. This constant cycle of information allows me to know when to sit back and leave them to it, or sit with them and help guide. And if I find someone on the team who cannot be trusted despite all efforts? Sorry — they’re out. Harsh, but fairer to the team as a whole. One bad apple should not ruin the success of the rest of the team.

To me, trust in the workplace is earned in the interview and cemented by doing good work, not something that still has to be earned by someone who has already been through extensive interviews, worked for the company for a number of months and is delivering on target. This is the same as how we live our lives. We sit on a chair because we trust it won’t break on us (unless our initial assessment of the chair shows it’s already broken), we eat food and trust it won’t kill us (unless it already smells bad), we fall in love trusting that our partners will be faithful, only removing that trust if they show otherwise.

Trust is good. My advice to my friend — go somewhere where you are trusted.

Debugging multiple Azure Functions apps at the same time

Tue, 13 Nov 2018 17:52:15 +0000

Dentists don’t want you to know these 5 tips that show how easy it is to run multiple Azure Functions apps locally and debug them all at the same time!

Number 3 will shock you!

Enough ClickBait nonsense, on to the post.

I recently built a demo app that has two Azure Functions projects in it - one using the new V2 Functions runtime and written in C#, and one using the old V1 runtime so that it can access some .NET framework bits and written in F#.

One of the really cool features that Azure Functions offers is the ability to run your functions locally with a full debugging experience, and this is supported in Visual Studio 2017 as well as VSCode.

This post focuses on how to do this in VS2017. For VSCode, follow these instructions.

This post looks at:

Debugging one function app
How to run multiple function apps at the same time
How to debug multiple functions apps at the same time using Visual Studio 2017

Debugging one Function app

This is easy to try out yourself, just create a new Azure Functions app in Visual Studio, add a breakpoint in the HTTP trigger then start debugging. The Azure Function app will launch in a local functions host, and your trigger will be available locally on http://localhost:7071/api/Function1 - so listening on port 7071. You can launch your favorite browser, navigate to this URL and your breakpoint will be hit, giving a full debugging experience with watch, edit and continue and all the normal bells and whistles.

Create one of these:

Use these settings (V2, HTTP Trigger, Anonymous):

Stick a break point in the function:

Start debugging the function the same way as any normal app (you may need to grant firewall access). This will launch a console window with the functions host running - you’ll see some cool ASCII art then the functions output including the HTTP URLs for all HTTP triggers:

Now launch your favorite browser and navigate to the Function1 URL. The function will run and your breakpoint will be hit. This is a full debug session with edit and continue:

Click Continue and see the output of the function in your browser:

There is also a storage emulator that you can run locally so you can read/write to blobs, tables and queues and even use triggers. Read more here.

Running multiple Function apps at the same time

Running one function app locally is cool, running multiple is even cooler!

Add a new function app to your project, then run it without debugging using Debug->Start without debugging. This will spin up the local Azure Functions host and start listening on port 7071.

Now the beady-eyed amongst you will notice this is the same port as the first functions app. By default the local functions host listens for HTTP requests on port 7071, so if you now launch the first functions app it will fail - with a really weird error:

Cannot access a disposed object.
Object name: 'IServiceProvider'.

The fix for this is to tell the functions host to run on a different port, and this can be done by adding arguments to the command line that is run when the function app is debugged. Kill all the functions hosts, and open the properties for the second function app. Head to the Debug tab, and set the Application arguments to be:

host start --pause-on-error --port 5860

This will start the functions host listening on port 5860. You can use any port you wish for this of course.

These settings are not saved in the project itself, just in your user settings so this won’t be in source control.

Now if you run the second function app it will be listening on a different port:

Debugging multiple function apps in VS2017

Running multiple is good, but debugging them is even gooderer!

Although Visual Studio 2017 can’t launch multiple apps at the same time, you can launch one then attach to the others. To do this:

Put break points in the HTTP triggers for both function apps
Start the first function app without debugging
Start the second function app through the debugger
Attach the debugger to the first function app:
- Select Debug->Attach to process…
- Search for func
- One func.exe process will be greyed out - this is the one that is currently running through the debugger. Select the other one and click Attach.

You should now be able to open http://localhost:7071/api/Function1 in your browser and hit the breakpoint in your first function app, and open http://localhost:5860/api/Function1 to hit the breakpoint in the second function app.

You don’t get edit and continue in the function app you attached to, only in the one run through the original debugger. If you want edit and continue make sure the function app you want to use this for is run through the debugger, and attach to the others.

Want to learn more about Azure Functions? Well we have a learning path for that as part of Microsoft Learn. Check it out here.

Updating your google-service.json from an AppCenter environment variable

Tue, 25 Sep 2018 14:15:50 +0000

I’ve been working on an app that needs push notifications, but also needs to be open source. This led me to a problem - the google-services.json file. This file is needed to sign up for Firebase messaging to implement push on Android, but it also contains my API keys, values I do NOT want to be public.

So - what to do?

Well I’m building and distributing my app using AppCenter and the build configuration contains a couple of useful settings that I can take advantage of - build scripts and environment variables.

I can define the contents of my google-services.json file inside an environment variable, add a dummy version to my GitHub repo, then overwrite the dummy one with the environment variable at build time. This means my secrets are safe, and the real value is protected using my AppCenter account. I can then also use the right version locally by excluding updates to this file from git as I’ve described before.

This works not only for an open source project, but if I wanted to have different settings for different builds (such as different push end points for QA or beta builds).

Check a dummy `google-services.json` file into source control

Add the google-services.json file to your project as normal. Blank out the file and add it to git. Revert the blanked out file and hide updates from git using:

git update-index --assume-unchanged ./google-services.json

Run this from the root of your Android app.

Define the environment variable

From the Build configuration tab of your build in AppCenter, turn on Environment variables. Add a new variable called GOOGLE_JSON and paste in the contents of your google-services.json file.

Create a build script to update the file

AppCenter can run build scripts before or after the build. These scripts have to have a specific name to be picked up by the build configuration. To create a pre-build script, create a script file in the root of your repo called appcenter-pre-build.sh. Add the following code to this script:

GOOGLE_JSON_FILE=$APPCENTER_SOURCE_DIRECTORY/<path>/google-services.json

if [ -e "$GOOGLE_JSON_FILE" ]
then
 echo "Updating Google Json"
 echo "$GOOGLE_JSON" > $GOOGLE_JSON_FILE
 sed -i -e 's/\\"/'\"'/g' $GOOGLE_JSON_FILE

 echo "File content:"
 cat $GOOGLE_JSON_FILE
fi

Update to be the path to the google-services.json file from the root of your repo. The $APPCENTER_SOURCE_DIRECTORY environment variable points to where on the build VM App Center has put your code.

This code will check for the file, and if it finds it copy the value of the environment variable into this file, followed by using a sed command to strip out escape characters as environment variables get double quotes escaped to \".

Once you add this file to your repo, it will appear under the Build Scripts in the Build Configuration.

If you Save and build the project, you will see the google-services.json file written to and dumped to the build log.

You can read more about the capabilities of AppCenter on docs.com.

Binding the Cognitive Services Android Speech SDK - Part 3 - Java 8 fun

Mon, 10 Sep 2018 11:22:55 +0000

In the first part of this post, I showed how to get started binding the Microsoft Cognitive Services speech API. In the second part I showed how to make the code look more C#-like. In this part, I’ll show how to use it and fix up a nasty issue with the Android compiler and using jars created with the latest versions of Java.

Using the SDK

To use the SDK, you will need an Android app. Create a new single-view Android app, and reference the SDK binding project. Then build the app and try to run it.

Then marvel, as your app spectacularly fails to compile with a really weird error message.

COMPILETODALVIK : Uncaught translation error : com.android.dx.cf.code.SimException: invalid opcode ba (invokedynamic requires --min-sdk-version >= 26)

WooHoo, invalid opcode ba. Ba indeed! What is this gibberish?

Well the issue comes down to Java versions. Android in the past only supported Java code up to version 7. They are now adding support for later versions but Xamarin doesn’t have this yet, and this is only available on newer versions of Android (>= 26). To make your code work on earlier versions and with Xamarin you have to do a thing called desugaring (yes, really), and this alters the Java bytecode to convert Java 8 bytecode to a version that is supported by Java 7.

At the moment there isn’t a nice IDE way to turn on desugaring, instead it has to be set inside the .csproj file of the client application. Open up the .csproj file for your newly created Android app inside VSCode (other editors are available, but hey - why would you), or by editing the file inside Visual Studio, and add the following to the default PropertyGroup:

<AndroidEnableDesugar>true</AndroidEnableDesugar>

Your app should now build without errors!

I have this working and compiling in the preview versions of Visual Studio on Windows at the time of writing cos that’s how I roll. If you are on stable and get weird errors then try with preview as I know support for this is being actively worked on.

If you do this on VS for Mac then you will get a crash at run-time. The workaround is documented here: https://github.com/xamarin/xamarin-android/pull/1973

Buiding an app using the SDK

To use the SDK you do need to sign up for the Speech service in Azure. Head to portal.azure.com and add a new Speech resource (at the time of writing this is in preview).

Once you have this, note down the endpoint from the Overview page. It will be a URL, and you will need the bit before .api.cognitive.microsoft.com. For example, if your endpoint is https://northeurope.api.cognitive.microsoft.com/sts/v1.0, then you will need northeurope. You will also need one of the two keys from the Keys page.

You can then create a SpeechFactory using these values:

var factory = SpeechFactory.FromSubscription(<SpeechApiKey>, <endpoint>);

Once you have a speech factory, you can create different recognizers - simple speech, a translator, or an intent recognizer using LUIS. To detect speech, handle the relevant events. You can see an example of using the TranslationRecognizer to convert English to spoken German in an example project in my GitHub repo.

Had a successful day. Created a #Xamarin binding for the @Azure #CognitiveServices Android speech SDK, and built a sample app that translates me voice into spoken German. pic.twitter.com/Bg4XDvhBjv
— Jim Bennett ☁️ (@jimbobbennett) August 31, 2018

In these three posts you have seen how to create a binding library for the Speech SDK aar, make the code more C#-like, then finally use it from a client app, working around a Java bytecode issue. You can check out my implementation and a sample at on GitHub. As always, the best source of information with much more depth is the java binding dos on docs.microsoft.com.

Let me know what you build with this SDK - my DMs are always open on Twitter.

Binding the Cognitive Services Android Speech SDK - Part 2, making the code more C#-like

Mon, 10 Sep 2018 11:22:44 +0000

In the first part of this post, I showed how to get started binding the Microsoft Cognitive Services speech API. In this part I show how to make the code look more C#-like. In the third part I’ll show how to use it and fix up a nasty issue with the Android compiler and using jars created with the latest versions of Java.

Making the namespaces more C#-like

The namespaces that come from Java are different from traditional C# style namespaces. Java namespaces are reverse-URL format, so the default one created for this project is Com.Microsoft.Cognitiveservices.Speech.Internal. A more-C# like one wouldn’t have the Com at the start. The default namespace also capitalizes the first letter of each part, but obviously doesn’t know what other letters should be capitalized - in this case the S in CognitiveServices.

The namespaces can be fixed by another attr entry in the Metadata.xml file - this time with the name set to managedName and the path set to the namespace. You will need one attr node for every namespace inside the Java library - so for com.microsoft.cognitiveservices.speech as well as the internal, util, intent and translation sub-namespaces. Change the namespaces to be more C# like, for example to Microsoft.Azure.CognitiveServices.Speech.*.

<attr path="/api/package[@name='com.microsoft.cognitiveservices.speech']" name="managedName">Microsoft.Azure.CognitiveServices.Speech</attr>
<attr path="/api/package[@name='com.microsoft.cognitiveservices.speech.internal']" name="managedName">Microsoft.Azure.CognitiveServices.Speech.Internal</attr>
<attr path="/api/package[@name='com.microsoft.cognitiveservices.speech.util']" name="managedName">Microsoft.Azure.CognitiveServices.Speech.Util</attr>
<attr path="/api/package[@name='com.microsoft.cognitiveservices.speech.intent']" name="managedName">Microsoft.Azure.CognitiveServices.Speech.Intent</attr>
<attr path="/api/package[@name='com.microsoft.cognitiveservices.speech.translation']" name="managedName">Microsoft.Azure.CognitiveServices.Speech.Translation</attr>

Once these namespaces are changed, your code will no longer build as the StdMapWStringWStringMapIterator class you added to the Additions folder will be using the old namespace, so fix this one up manually. Your code should then now build.

Handling code with callbacks

The Speech SDK is actually implemented as cross-platform C++ code, with some platform-specific C++ code added to support each platform. This code is than wrapped using SWIG to make it available as Java code for Android, C# code for Windows etc.

This model has the problem that events are not implemented as standard Java listeners. If they were, then the binding library would automatically convert them to C# events. Seeing as they are not, you will need to convert them to events manually.

This is a very specific example for this one library, so it is unlikely that other libraries will need exactly the same code - instead I thought I’d write about it as an example of the kind of thing you may have to do to make the code more C# like.

How events are implemented in this SDK

In this SDK, events are implemented by passing an object that implements the IEventHandler interface to the AddEventListener method on a property of type EventHandlerImpl. When the event is raised, the OnEvent method on the IEventHandler interface is called.

For example, in the SpeechRecognizer class there is a property called FinalResultReceived of type EventHandlerImpl, and you subscribe to this ’event’ by passing an instance of IEventHandler to the AddEventListener on this property.

This pattern is not idiomatic C#, and is annoying to use as you will need to declare a class that implements the IEventHandler interface just to handle the event.

Making this code more C#-like

To make this code more C# like, what you can do is:

Create a generic implementation of IEventHandler that raises an event
Add a C# event to the class has a property of type EventHandlerImpl
Add an instance of the IEventHandler implementation to this class, and add this as a listener to the EventHandlerImpl property
In the C# event, explicitly implement the add and remove methods. In these methods, add or remove the event from the event on the IEventHandler implementation
Hide the EventHandlerImpl property from client code by marking it internal

Creating the generic event handler

To create the event handler, add a new class to the Additions folder called EventMapper. The code for this is:

class EventMapper<T, T1> : EventMapper, IEventHandler
 where T : class
 where T1 : class
{
 readonly object sender;
 readonly Func<T1, T> argExtractor;
 public EventMapper(object sender, Func<T1, T> argExtractor)
 {
 this.sender = sender;
 this.argExtractor = argExtractor;
 }

 public event EventHandler<EventArgs<T>> EventRaised;
 public void OnEvent(Java.Lang.Object p0, Java.Lang.Object p1)
 {
 EventRaised?.Invoke(sender, new EventArgs<T>(argExtractor(p1 as T1)));
 }
}

This is an internal class, so is only available to the binding library.

This class has two parts - an event and an argExtractor.

The event is a standard C# event using the EventHander<> delegate type - so when called it passes the sender as an object and some event arguments that derive from EventArgs. The EventArgs<T> type is not one that exists in .NET Standard (which I am very surprised about as I’ve created this so many times, as have others). You will need to implement this yourself, so add a class called EventArgs with the code below. This event args class is a simple wrapper for a value that needs to be passed to the event and saves you creating a load of custom event arg classes for each different value type that you want to pass.

public class EventArgs<T> : EventArgs
{
 public T Value { get; }
 public EventArgs(T value)
 {
 Value = value;
 }
}

When events are handled by the SDK, it passes it’s own event args implementation containing a value for those args. For example, the FinalRecognitionResult event handler on the SpeechRecogniser is passed an instance of SpeechRecognitionResultEventArgs, containing a Result property of type SpeechRecognitionResult. These event args don’t derive from the standard .NET EventArgs class, so you need a way to extract the relevant value and populate that into an EventArgs class, and this is what the argExtractor does - it takes the SDK args and pulls out the value needed. This is then wrapped in an EventArgs<T> and passed to the event invocation.

Handling an event

In the SpeechRecogniser class there is a FinalRecognitionResult handler that raises an event passing an instance of SpeechRecognitionResultEventArgs, containing a Result property of type SpeechRecognitionResult. To map this to C#, you would add another part to the SpeechRecogniser class:

public partial class SpeechRecognizer
{
}

You would then add a field for the event mapper:

EventMapper<SpeechRecognitionResult, SpeechRecognitionResultEventArgs> finalResultMapper;

Then you add a C# event for the final result:

public event EventHandler<EventArgs<SpeechRecognitionResult>> FinalResult
{
 add {}
 remove {}
}

In the add method, if the EventMapper hasn’t been created yet, you create it and pass it to the AddEventListener of the bound handler. When it is created you will need to pass in the sender which is passed to the events when invoked, and this is always this. You also need to pass in a mapper function to extract the SpeechRecognitionResult from the SpeechRecognitionResultEventArgs, which can be a simple lambda function to return the Result property. Then you subscribe the passed in value to the event on the mapper.

add
{
 if (finalResultMapper == null)
 {
 finalResultMapper = new EventMapper<SpeechRecognitionResult, SpeechRecognitionResultEventArgs>(sender, e => e.Result);
 finalResultMapper.AddEventListener(handler);
 }
 finalResultMapper.EventRaised += value;
}

For the remove function, if the mapper has been created you can unsubscribe the value from the event:

remove
{
 if (finalResultMapper != null)
 finalResultMapper.EventRaised -= value;
}

This is a lot of boilerplate code, so in my version I refactored this into some static methods. You can see these in my GitHub repo.

Hiding the original event handler

Now that you have C# style events, it is cleaner to hide the old event handler implementation to stop client code from calling instead of your nice, shiny, C# events. To do this, you can use an attr in the Metadata.xml file to change the visibility of the properties for the old event handlers to internal. Seeing as these are the only places that IEventHandler and EventHandlerImpl are used, you can also mark these as internal. That way if you leave any EventHandlerImpl properties as public, the compiler will give you an error - a great way to ensure you have mapped all the events.

To mark these as internal, grab the paths from the source files in the obj folder, and add an attr node with the name set to visibility, and the content of the node set to internal. The code below shows this for the FinalResultReceived property on the SpeechRecognizer, as well as the IEventHandler and EventHandlerImpl classes. Repeat this for all the event handler properties across all classes.

<attr path="/api/package[@name='com.microsoft.cognitiveservices.speech']/class[@name='SpeechRecognizer']/field[@name='FinalResultReceived']" name="visibility">internal</attr>

<attr path="/api/package[@name='com.microsoft.cognitiveservices.speech.util']/class[@name='EventHandlerImpl']" name="visibility">internal</attr>
<attr path="/api/package[@name='com.microsoft.cognitiveservices.speech.util']/class[@name='IEventHandler']" name="visibility">internal</attr>

In the final part, I’ll show how you can call this code from a client app, as well as fixing up a nasty issue with the Android compiler and using jars created with the latest versions of Java. You can find the code for this in my GitHub, and you can read more on docs.microsoft.com

Binding the Cognitive Services Android Speech SDK - Part 1, binding the library

Sun, 09 Sep 2018 14:55:00 +0000

As part of the Microsoft Cognitive Services speech API, there is a native Java Android SDK available as an .aar file. I wanted to use this in a Xamarin app, so I created a binding project for it.

The code for this is available in my GitHub.

Binding an SDK is a four step process:

Create the binding project with the relevant jar or aar file
Make any necessary tweaks to the code or project to make it compile
Make any required amendments to the code to make it into idiomatic C#
Test it all out and fix up any issues

There are some great docs available on the basics for doing this at docs.microsoft.com, but each library is different and can have it’s own unique challenges so I thought I’d write a few posts to highlight the steps I needed to take to bind the speech SDK.

In this first part, I’ll show how to create a binding project, add the speech SDK aar file, and make everything compile. In the second part I’ll show how to make the code more idiomatic C#, then in the third part I’ll show how to use it and fix up a nasty issue with the Android compiler and using jars created with the latest versions of Java.

Binding the SDK

The first step was to create a binding project and add the .aar file. I followed the instructions in the Xamarin docs, creating a new Android binding project and adding the client-sdk-0.6.0.aar file to the Jars folder.

When you compile this project, the compile step will generate code to bind every Java class it finds. Each class in the generated code is a wrapper for a Java class - it doesn’t re-implement the Java code, it instead creates the same kind of thin wrapper that is used by the Xamarin Android SDK bindings. If you want to see this generated code, you can find it in the obj/${Congfiguration}/generated/src folder.

Making it work

After doing this, I compiled the library and hit a couple of compiler errors that I needed to fix up:

/.../Com.Microsoft.Cognitiveservices.Speech.Internal.StdMapWStringWString.cs(72,72): Error CS0738: 'StdMapWStringWString' does not implement interface member 'IIterable.Iterator()'. 'StdMapWStringWString.Iterator()' cannot implement 'IIterable.Iterator()' because it does not have the matching return type of 'IIterator'. (CS0738) (Speech)

/.../Com.Microsoft.Cognitiveservices.Speech.Internal.StdMapWStringWStringMapIterator.cs(83,83): Error CS0738: 'StdMapWStringWStringMapIterator' does not implement interface member 'IIterator.Next()'. 'StdMapWStringWStringMapIterator.Next()' cannot implement 'IIterator.Next()' because it does not have the matching return type of 'Object'. (CS0738) (Speech)

The reason for this is that the Xamarin Java SDK doesn’t contain the generic versions of IIterator and IIterable which are used by this library, instead it will default to using the non-generic versions, and the implementation of the generic interfaces doesn’t match the signature of the non-generic version. So - how can it be fixed?

Metadata.xml and Additions

Inside the binding project you can both alter the generated code and add new code.

Transforms/Metadata.xml - in this XML file you can alter the code that is generated. You can add entries to this file to remove some of the autogenerated code, either at class, method or property level. You can also change the generated code, for example changing the namespace - something especially useful to change from Java style namespaces to C# style.
Additions - In this folder you can add code that is compiled into the final dll. Each autogenerated class is declared as partial, so not only can you add new classes and code, you can also add new parts to a generated class.

Fixing the `StdMapWStringWString` code

The StdMapWStringWString class implements a generic version of the IIterable interface - IIterable<StdMapWStringWStringMapIterator>. The Xamarin Java SDK doesn’t contain the generic base interface, so the bound library defaults to implementing IIterable. The problem is this interface contains a method Iterator that returns a different type in the generic version to the non-generic version. The generated code implements this method returning a StdMapWStringWStringMapIterator, but the non-generic version expects a method returning IIterator, so you get a compiler error.

This is simple enough to fix - you just need to change the return type of the binding to be IIterator, and this can be done in the Metadata.xml file.

The Metadata.xml file is a file containing transforms that you want to make to the generated code - and can be adding new items, removing items or change the attributes of items such as the name or the return type.

Open the generated code from the obj/${Congfiguration}/generated/src/ folder - the file will be called Microsoft.Azure.CognitiveServices.Speech.Internal.StdMapWStringWString.cs. If you look at all the public items in this file (the class, public methods and public properties), you will see each one has a comment describing the Metadata.xml path:

// Metadata.xml XPath class reference: path="/api/package[@name='com.microsoft.cognitiveservices.speech.internal']/class[@name='StdMapWStringWStringMapIterator']"
public partial class StdMapWStringWStringMapIterator
 ...

This path is used to identify each item to the Metadata.xml file, so locate the Iterator() method and note the path.

Open the Metadata.xml file and add a new attr node inside the metadata node. Set the path attribute of this node to match the path in the comment for the Iterator() method. Then add an attribute called name with the value managedReturn to tell the transformations that this is a changed to the managed return type - so the type for the binding library only. This will treat the underlying return value as the original type which is what you want. The value for this attribute is set inside the node, and should be Java.Util.IIterator.

The full node is shown below:

<metadata>
 <attr path="/api/package[@name='com.microsoft.cognitiveservices.speech.internal']/class[@name='StdMapWStringWString']/method[@name='iterator' and count(parameter)=0]" name="managedReturn">Java.Util.IIterator</attr>
</metadata>

Now if you compile the project, one error will be gone. If you re-open the generated file you will see the new return type.

You can read more on the capabilities if this file in the docs.

Fixing the `StdMapWStringWStringMapIterator` code

The Xamarin Java SDK doesn’t contain the generic version of IIterator, so the bound code uses the non-generic version. This interface has a Next() method that returns the next item from the collection being iterated. In the non-generic version of this interface, Next() returns a Java.Lang.Object, whereas in the generic version it returns the generic arg, in this case a Java.Lang.String. This means the generated code uses the non-generic interface, but the implementation uses the generic method, causing a compiler error.

The fix for this is a little bit more work - you can’t just change the return type as it is used in a private method created by the binding. Instead, the fix for this is to remove the generic Next() method and replace it with a non-generic version. Re-writing binding methods is not easy as there is a lot of code in the binding, so for cases like this the best way is to copy the generated code, adjusting it to suit. If you open the com.microsoft.cognitiveservices.speech.internal.StdMapWStringWStringMapIterator.cs file from the obj directory, and look at the Next() method you will see the implementation consists of not just the Next() method, but also an n_next() method, a GetNextHandler() method and a cb_next Delegate field. This is all the plumbing needed to create the binding method and call through to the underlying Java method.

Adding the new method

Lets’s start by adding the new method as this will mostly be a copy of the existing code. Add a new class to the Additions folder called StdMapWStringWStringMapIterator, mark the class as partial and change the namespace to match the generated file (these will be fixed up later to be more C#-like).

namespace Com.Microsoft.Cognitiveservices.Speech.Internal
{
 public partial class StdMapWStringWStringMapIterator
 {
 }
}

Copy the Next(), GetNextHandler(), n_next() methods and the cb_next field from the generated code and paste them into the new class part. Strip out any unnecessary namespaces and use var everywhere - this will make it easier when you change the namespaces to be more C#-like in the next post.

Change the return type of Next to be Java.Lang.Object instead of Java.Lang.String. Leave the types in the Register attribute and __id fields as they are, as the underlying method that is called returns a Java.Lang.String, and you only need to change the return type for the binding wrapper.

In the n_next() method, change the return call to call ToString() on the result of the call to Next() to use the correct type. The final code will look like this:

namespace Com.Microsoft.Cognitiveservices.Speech.Internal
{
 public partial class StdMapWStringWStringMapIterator
 {
 static Delegate cb_next;
# pragma warning disable 0169
 static Delegate GetNextHandler()
 {
 if (cb_next == null)
 cb_next = JNINativeWrapper.CreateDelegate((Func<IntPtr, IntPtr, IntPtr>)n_Next);
 return cb_next;
 }

 static IntPtr n_Next(IntPtr jnienv, IntPtr native__this)
 {
 var __this = Object.GetObject<StdMapWStringWStringMapIterator>(jnienv, native__this, JniHandleOwnership.DoNotTransfer);
 return JNIEnv.NewString(__this.Next()?.ToString()); // ToString called on the object.
 }
# pragma warning restore 0169

 [Register("next", "()Ljava/lang/String;", "GetNextHandler")]
 public virtual unsafe Java.Lang.Object Next() // Return type changed from string to object
 {
 const string __id = "next.()Ljava/lang/String;";
 try
 {
 var __rm = _members.InstanceMethods.InvokeVirtualObjectMethod(__id, this, null);
 return JNIEnv.GetString(__rm.Handle, JniHandleOwnership.TransferLocalRef);
 }
 finally
 {
 }
 }
 }
}

Removing the `Next` method

To remove a method from the autogenerated file, you add a remove-node entry to the Metadata.xml file with the path of the Next() method node you want to remove. To remove the Next() method, add this following to this file:

<remove-node path="/api/package[@name='com.microsoft.cognitiveservices.speech.internal']/class[@name='StdMapWStringWStringMapIterator']/method[@name='next' and count(parameter)=0]"/>

The syntax is <remove-node path="..."/> where the path comes from the comment in the generated code. Once this line has been added, compile the code and check the generated file in the obj directory. The compiler error about the missing Next() method will have gone, and the Next() method will be removed from the autogenerated file - when the library is built it will use the version in the file in the Additions folder.

You should now be able to compile this library successfully.

In the second part, I’ll show how you can make the code more C#-like. You can find the code for this in my GitHub, and you can read more on docs.microsoft.com

Running AI models on iOS, Android and Windows using Xamarin

Fri, 20 Jul 2018 16:59:01 +0000

I created a NuGet package a while ago to allow you to run models exported from the Azure Custom Vision service on iOS and Android in Xamarin apps from your cross-platform code. You can read about this here.

Since then, the Custom Vision service has added ONNX export, meaning you can now run these models on-device on Windows as well. This meant it was time to update my plugin to support Windows.

This was released in v2.0 of the plugin - I thought I’d be a good developer and bump the major version number as I’ve broken the API.

API changes

The main change is that the IImageClassifier.Init method has now gone. When I wrote this it was a bit of a fudge - it took a single model name and a model type. The model type was only needed on Android as some image adjustments needed to be made based on the model type used. It also meant that there was no way to use a different TensorFlow labels file, it had to be called labels.txt. With Windows, the Init call would need a list of labels, so it made sense to strip it down and have platform-specific calls to allow each platform to be passes just what is needed.

Initialization on iOS

Models can be compiled before being used, or compiled on the device. To use a pre-compiled model, compile the downloaded model using:

xcrun coremlcompiler compile <model_file_name>.mlmodel <model_name>.mlmodelc

Add the model (compiled or uncompiled) to the Resources folder in your iOS app. Then initialize the plugin using:

iOSImageClassifier.Init("<model_name>");

Passing in the name of the model without the mlmodel or mlmodelc extension. If the model is uncompiled, it will be compiled before use.

Initialization on Android

Add the model.pb and labels.txt files to the Assets folder of your Android app. Then initalize the plugin using:

AndroidImageClassifier.Current.Init("model.pb", "labels.txt", ModelType.General);

These values are actually defaults, so you can leave these off and just use AndroidImageClassifier.Current.Init() if you want.

A note about model types. You needed the model type because the original TensorFlow export created models that needed image adjustments for some model types to work correctly. Since 7th May 2018 then models have a layer that adjusts for this automatically. The plugin will detect this and only make image adjustments if needed, so the model type is only relevant for models created before this date. It is ignored for models created after.

Initialization on Windows

Add the <model>.onnx file to the Assets folder of your Windows app. A <model>.cs file will be created to process the model in the root of your UWP app. The wrapper will have a class called <model>ModelOutput and in the constructor for this class will be some code to create a dictionary called loss:

this.loss = new Dictionary<string, float>()
{
 { "<label 1>", float.NaN },
 { "<label 2>", float.NaN },
 ...
};

This defines the labels in the correct order for the model. These labels will need to be passed to the Init method in the correct order.

The Init method on Windows is also an async method, so will need to be called from another async method, such as by overriding OnNavigatedTo on a page and marking it as async. Await the init method passing the model name and labels like this:

await WindowsImageClassifier.Init("Currency", new[] { "<label 1>", "<label 2>", ... });

I’m sure this can be improved, including adding code to detect the labels from the ONNX model, so feel free to raise a PR with any improvements. All the code is on GitHub here.

Running custom vision models on a Windows device

Fri, 20 Jul 2018 16:29:53 +0000

Recently I wrote about creating AI models using the Azure Custom Vision Service. In these posts I looked at creating and training models, running them online, then finally exporting the models to run on iOS using CoreML and Android using TensorFlow.

Recently Microsoft announced another way to export models - as ONNX models that can be run using Windows ML. Like with CoreML and TensorFlow, these are models that can be run on-device, taking advantage of the power of the devices GPU instead of needing to be run in the cloud.

ONNX models can be exported in the same way as CoreML and TensorFlow - select you iteration, click the Export button to generate the model, then the Doanload button to download it.

To use this model you need a UWP app targeting Build 17110 or higher of the Windows SDK (as this version is the first one containing the Windows ML API). Create a new app, then drag your downloaded model into the Assets folder. When you add the model, some wrapper classes will be created inside a file in the root directory of your app with a really weird name - they will all start with the Id of your model converted into a format that is C# friendly. The three classes that will be created are for inputs, outputs and running the model - and will end with ModelInput, ModelOutput and Model.

It would make sense to rename them, so rename the file to something sensible that reflects your model, then rename the classes as well to ModelInput, ModelOutput and Model.

ModelInput is the input to the model and contains a single property called data which is a VideoFrame. You can populate this with a capture from the camera or a frame from a video feed.
ModelOutput is the outputs of the model as a dictionary of strings to floats. The dictionary is auto-populated with the tags from the model, and when the model is run this will be updated to set the percentages for each tag.
Model is the class that evaluates the model, binding the inputs and outputs and executing the model.

To create a model, use the static CreateModel method on Model, passing in the <model>.onnx file as a StorageFile, loaded like this:

var file = await StorageFile.GetFileFromApplicationUriAsync(new Uri($"ms-appx:///Assets/<model>.onnx"));

To run the model, capture a VideoFrame, set it on an instance of ModelInput, then pass that to the EvaluateAsync method on the model. This will return a ModelOutput with the probabilities set.

You can find a sample of this in the GitHub Azure Samples repo. You can also learn more by checking out the docs.

Setting iOS permission descriptions in Visual Studio 2017

Thu, 05 Jul 2018 14:39:40 +0000

I spend most of my developer day using Visual Studio for Mac, but occasionally I flip back to Visual Studio 2017 on Windows. Last time I flipped back I tried to build an app that uses the camera and I got stuck on one simple thing - setting the camera usage description.

If you double click on the info.plist file in an iOS app project, it will open in the iOS manifest editor. This editor allows you to set or change a number of settings for your app, such as the bundle identifier, app name or scheme URLs that your app uses. But one thing this editor doesn’t support is setting the usage descriptions.

After a quick chat to some other developers, I found out how to do it (thanks Pierce).

From Visual Studio, right-click on the info.plist file, then select Open With….

From the Open With… dialog, select the Generic PList Editor.

Once the file has been opened, click the + on the bottom row to create a new property. Click the property name to get a drop down showing all the supported properties, then click the one you want.

Finally add the description to the Value column.

What the @ - fixing weirdness in return JSON from Azure functions using F#

Tue, 19 Jun 2018 15:59:44 +0000

I’ve been playing a lot with F# recently, both to build Xamarin apps using Elmish.XamarinForms and for some Azure Functions. Whilst building an HTTP trigger I came across some weirdness when serializing a record type to JSON.

This is the relevant parts of my code:

type Output = { TotalBalance : float }
let Run(req: HttpRequestMessage, boundTable: IQueryable<Transaction>, log: TraceWriter) =
async {
// stuff
return req.CreateResponse(HttpStatusCode.OK, { TotalBalance = sum })
} |> Async.RunSynchronously

Now it’s normal to assume the Output type would be serialized to JSON correctly, leading to something like:

{
 "balance" : 100
}

Unfortunately not - what I actually get is:

{
 "Balance@" : 100
}

Note the weird @ symbol on the property name.

So I went spelunking around Google and SO for a fix. The first suggestion was to add the [<CLIMutable>] attribute to my Output type, but this didn’t actually work, the @ symbol was still there.

In the end I found the fix here: https://stackoverflow.com/questions/43118406/return-an-f-record-type-as-json-in-azure-functions/48718297#48718297. You can pass a different JSON formatter to the CreateResponse call, and configure that to return the correct property names:

let jsonFormatter = Formatting.JsonMediaTypeFormatter()
jsonFormatter.SerializerSettings.ContractResolver <- CamelCasePropertyNamesContractResolver()
return req.CreateResponse(HttpStatusCode.OK, { TotalBalance = sum }, jsonFormatter)

Done!

Describing a photo in a mobile app using Azure Computer Vision

Wed, 30 May 2018 16:05:41 +0000

I recently gave an introduction to Xamarin talk at Imperial College, London and wanted to build a cool app to show off what you can do on mobile using the awesome Cognitive Services available on Azure. I only had about 30-40 minutes to not only introduce Xamarin, but build an app so I decided to throw together a simple app to take a photo and describe it using the Azure Computer Vision service.

It really is simple to set up and use this service. Head to the Computer Vision cognitive services site, click the big Try the Computer Vision API button. Log in with an appropriate provider, and get an API key, noting the region the key is for.

From inside your Xamarin app, install the pre-release Microsoft.Azure.CognitiveServices.Vision.ComputerVision NuGet package into all the projects. Then install the Xam.Plugin.Media NuGet package and follow the instructions in the readme.txt that is auto-opened to enable permissions and other gumpf.

Add some code to take a photo using the media plugin:

var opts = new Plugin.Media.Abstractions.StoreCameraMediaOptions();
var photo = await Plugin.Media.CrossMedia.Current.TakePhotoAsync(opts);

Next, set up the Computer Vision API by creating an ApiKeyServiceClientCredentials with your API key, then constructing an instance of ComputerVisionAPI using these credentials, not forgetting to set the region.

 var creds = new ApiKeyServiceClientCredentials("<your key here>");
 var visionApi = new ComputerVisionAPI(creds)
 {
 AzureRegion = AzureRegions.Westeurope
 };

Finally, get a stream containing the image and pass it to the computer vision API.

var desc = await _visionApi.DescribeImageInStreamAsync(photo.GetStream());

You can then access a description for the image using the Captions property on the ImageDescription that is returned. You can also get a list of tags for the image using the Tags property. The image below shows my app using this to caption an image.

You can find the code for this on my GitHub, and you can read more on the computer vision service in the docs.

Xamarin in Action is now on the bookshelves!

Mon, 21 May 2018 09:03:04 +0000

Finally, after 2 years since I first put pen to paper (well, put keystroke to Atom), Xamarin in Action is done and available from your favorite bookseller as a printed, paper book, with a forward from my good friend, the amazing James Montemagno!

You can buy this book in one of two ways - an eBook in PDF, kindle, iBook format, or a printed book. If you buy the printed book, you get the eBook for free! It’s available from all good booksellers, or direct from Manning - and if you buy direct you can use code ‘xamarininaction’ for a hefty discount!

This book is designed to teach you how to build a production quality Xamarin app, from idea through implementation and testing, to the store and beyond. It doesn’t cover just a list of UI widgets with lists of properties and methods, instead it focuses on building real world, well written, testable apps using MVVM. It covers using traditional Xamarin for iOS and Android, looking at how to maximize the amount of cross-platform code in your app, how to build UIs using Android XML and iOS Storyboards, how to share and unit test your UI logic, and how to test your UI. It then shows how to use AppCenter to build, test and distribute your app, as well as monitor usage and crashes after publication.

Here’s some things that people have said about this book:

Buy your copy today - head to the Manning website and use code ‘xamarininaction’ for a hefty discount!

Xamarin Essentials

Tue, 08 May 2018 15:48:04 +0000

TL;DR - check out Xamarin Essentials in the official documentation.

Like a lot of developers, I love how Xamarin allows me to share business logic between iOS and Android apps, and share UI using Forms, but still have access to the native APIs. But one thing has been missing - a consistent, out of the box way of accessing native APIs from cross-platform code. The power of Xamarin is you have full API access allowing you to take advantage of the unique features of each platform, but sometimes you just want to do something in one line of code that applies to all platforms.

Plugins have helped with this - so you can install a NuGet package such as the permissions or connectivity plugins from James Montemagno and have a nice cross-platform API, but discoverability has always been a bit hard, you have to know there is a plugin and what the NuGet package is called.

To make it easier, there is now Xamarin.Essentials, a single package that you can install into any Xamarin app to get cross-platform access to a wide range of APIs such as accelerometer, compass, network connectivity, keeping the screen awake and more. You can see the full list of supported features on the Xamarin Essentials GitHub page. This package currently has 24 different API sets, with more planned. Essentials is for all Xamarin apps, both traditional and Forms apps.

I’ve started building an app to track my daughters pocket money - I don’t use cash so it’s hard to regularly put cash into a piggy bank, instead I though I’d write an app I can use to regularly put an amount into a virtual account, and when she wants to spend it, use my card for the purchase (easier with Amazon for example) and deduct the cost from her virtual piggy bank. The first part of this app is logging in, and for that I’m going to use social auth and Azure, but there is no point in showing a login screen if the user is not connected to the internet, so I thought I’d give Essentials a spin and use the connectivity part.

Setting it up is easy, just install the NuGet package into all projects in your app (it must be all as it has the platform specifics in the iOS and Android packages). For Android you do need to initialize the library and set up some permissions stuff in all activities. I’m using Xamarin Forms, so only have one activity to do it in:

protected override void OnCreate(Bundle savedInstanceState)
{
 ...
 Forms.Init(this, savedInstanceState);
 Xamarin.Essentials.Platform.Init(this, savedInstanceState);
 ...
}

public override void OnRequestPermissionsResult(int requestCode,
 string[] permissions,
 [GeneratedEnum] Permission[] grantResults)
{
 Xamarin.Essentials.Platform.OnRequestPermissionsResult(requestCode, permissions, grantResults);
 base.OnRequestPermissionsResult(requestCode, permissions, grantResults);
}

Done - Essentials is all set up. Checking connectivity is now really easy:

using Xamarin.Essentials;
...
if (Connectivity.NetworkAccess != NetworkAccess.Internet)
{
 ... // no internet access 
}

Checking network access is as simple as checking the value of the Connectivity.NetworkAccess static property. If this is set to NetworkAccess.Internet then there is internet access. There is also an enum member called ConstrainedInternet when there is a connection but with a poor signal. Otherwise - no internet.

These properties are all static, which might seem odd to developers who prefer interfaces for use in IoC containers. There is a discussion around this on the README.md on GitHub.

I might also want to be notified when the users internet access has changed. For example if I hide the login button when they are not connected due to being in airplane mode, I’d want my app to know when they turned airplane mode off so it can show the login button again. I can do this using the Connectivity.ConnectivityChanged event. This means the code for my login page can be:

public LoginPage()
{
 ...
 Connectivity.ConnectivityChanged += e => ShowOrHideLoginButton();
 ShowOrHideLoginButton();
}

void ShowOrHideLoginButton()
{
 LoginButton.IsVisble = (Connectivity.NetworkAccess == NetworkAccess.Internet ||
 Connectivity.NetworkAccess == NetworkAccess.ConstrainedInternet);
}

You can read more about Xamarin Essentials in the official documentation. Also check out the GitHub page - this library is fully open source like all Xamarin goodness, and you can raise issues or pull request there if you feel like contributing to this awesome project.

Building mobile apps in F# using Xamarin.Forms and Elmish

Sun, 06 May 2018 15:00:40 +0000

FSharp and Xamarin

Xamarin is well known for allowing you to build mobile apps in C#, but you can use F# as well. F# is fully supported by the compiler and toolchains in both Visual Studio 2017 and Visual Studio for Mac. It’s also pretty much supported by all the Xamarin tools including Xamarin.Forms (except for one bug that should be fixed soon).

This is great for F# fans like me, but one thing that has been missing for a while has been architecture recomendations. Mobile apps, like other UI heavy apps are very object oriented. Buttons are objects, labels are objects and they fit very nicely into the classic ‘OO’ space. This has meant that traditionally UI apps are also built using objects - it’s objects all the way down and mutable state everywhere. This doesn’t match well with functional programming paradigms which try to move away from objects and mutability.

To go functional when it comes to UI, one trend is towards Elm. This is an architecture that comes from Erlang and has become popular in the F# community via Elmish for Fable for web sites. This architecture abstracts the object nature of the UI away and replaced by a purely functional model. The app runs with a function to create a view, a function to initialise an immutable model, and an update function to handle messages from either the UI or other parts of the system. The update function takes the current model and a message, and returns a new instance of the model containing a copy of the original model updated based off the message recieved. Once a new model is generated the UI is updated.

This paradigm has been ported to F# by Don Syme as Elmish.XamarinForms, an open source project also available on NuGet. This project provides two possible implementations of Elmish for Xamarin.Forms - known as half-elmish and full-elmish. In this post I’ll look at the half-elmish implementation as this is more suited to Xamarin developers who want to move to F# (full-elmish is more suited to F# developers wanting to move to Xamarin).

Half-Elmish

Half-elmish mixes traditional Xamarin.Forms development with the elmish architecture. At it’s most basic, you define your UI using XAML, setting bindings on the UI elements as if you were using MVVM. You then implement an imutable model, an update method that handles messages that you define and returns an updated model, and a view method that configures the bindings for your UI including property bindings and commands. When a message is handled the bindings are re-evaluated and the UI updates using the normal Xamarin.Forms data binding.

Getting started

Start by creating a new blank Xamarin.Forms F# app using either Visual Studio 2017 or Visual Studio for Mac called HelloElmish. The latest versions of both will allow you to create apps with a .NET standard core project. Add the Elmish.XamarinForms NuGet package to all the projects in your app.

We’ll create a fairly boring app, one that tracks the number of clicks of a button. Not exciting, but enough to illustrate the basics to get you started. Open the MainPage.xaml file, remove the contents of the ContentPage and add the following:

<StackLayout>
 <Label Text="{Binding Path=[Count]}"/>
 <Button Text="Increment"
 Command="{Binding Path=[Increment]}"/>
</StackLayout>

Notice how the bindings are set using an array-indexer style syntax. This is part of the Elmish implementation of binding, so if you want to know more head to GitHub and dig through the source, but for now just accept that this is how we bind.

Next, lets implement the application logic. Open the App.xaml.cs file. We need to declare a model to track the button click count, an enum to define the messages our app supports, as well as three functions - init, update and view.

The model is fairly simple, so declare this before the App class declaration:

type Model =
{
Count : int
}

Next up, add an enum type for the messages that the update function will handle:

type Msg =
| Increment

Now we need to implement the init method. This is called when your app starts up to create the initial model, so add this function to the App class:

let init () = { Count = 0 }

This creates a new model with the count set to 0. Next is to add a function to handle a message to increment the counter. This takes a message and a model, so add this to the App class:

let update msg model =
match msg with
| Increment -> { Count = model.Count + 1 }

This function matches the message with the Increment message, and if it matches returns a new model with the Count set to the count of the previous model plus one. It may seem overkill to match when we only have one message type, but we’ll add another message type later. Notice how it is always a new model that is returned - the Model type is immutable so we always have to create a new copy.

After the update function we need to add the view function to the App type. This is used to create bindings between the UI, the model and the messages. This function returns a tuple of a Xamarin.Forms page and a list of bindings:

let view () =
MainPage(),
[ "Count" |> Binding.oneWay (fun m -> m.Count.ToString())
"Increment" |> Binding.msg Increment
]

You’ll need to open the following modules at the top of the file for this to compile:

open Elmish.XamarinForms
open Elmish.XamarinForms.StaticViews

This view method defines two bindings. One is a one way binding for the "Count" property that binds the value to a function that returns the Count from the model converted to a string. The next is a binding from the "Increment" command to the Increment message.

When the app is launched this view function is run and the page is created. The values of the bindings are evaluated and set. The oneWay binding sets the value on the Label based on the return value of the function, and the msg binding binds the button command property to a command created for you by the elmish implementation. Executing this command calls the update function passing in the Increment message and the current model.

When the button is clicked, this command is executed, calling the update function. The new model that comes back from this function call is then passed through the bindings again, and if the values returned from the bindings is different from the previous value the UI widget gets updated. In our case, the update method increments the count, so the value of the one way binding would increment from 0 to 1, and the UI would be updated to show 1.

The final thing to do is wire everything up to make the app kick off the elmish functions. Add the following to the App class:

let runner =
Program.mkSimple init update view
|> Program.withStaticView
|> Program.run
do base.MainPage <- runner.InitialMainPage

In addition, remove the setting of the MainPage when inheriting from App:

type App() =
inherit Application()

This code creates a runner that creates an elmish program using our init, update and view functions. It uses a static view that tells elmish that the page is created once and updated via binding. The full-elmish implementation doesn’t use XAML pages, instead it creates the UI on the fly and the whole UI is rebuilt after each update (with a lot of logic inside the elmish code to reuse stuff for performance). Full-elmish is outside the scope of this post.

This runner has an initial main page property that is set as the MainPage for your App class. The elmish architecture supports navigation and multiple pages, and I’ll cover this in a later blog post.

Build everything and run the app. You’ll be able to click the button and see the count increment.

Getting more advanced

Lets make this a bit more advanced by adding a decrement button. Start by adding a new button to the StackLayout:

...
<Button Text="Decrement"
 Command="{Binding Path=[Decrement]}"/>
...

Next add a message type for this:

type Msg =
| Increment
| Decrement

Then add code to the update function to handle this message:

let update msg model =
match msg with
| Increment -> { Count = model.Count + 1 }
| Decrement -> { Count = model.Count - 1 }

Then finally wire up the message in the view function:

let view () =
MainPage(),
[ "Count" |> Binding.oneWay (fun m -> m.Count.ToString())
"Increment" |> Binding.msg Increment
"Decrement" |> Binding.msg Decrement
]

Run the app and you’ll be able to increment and decrement the value.

The code for this is on GitHub.

Learn more

We’ve seen a very simple example of how to use F# to build Xamarin.Forms apps using the elmish architecture. For a more complicated example, check out my Tic Tac Toe game on GitHub or any of the Elmish.XamarinForms samples.

Also check out my video from F#Conf on using F# to build Xamarin apps below or on Channel9, and don’t forget there is plenty of official Xamarin F# documentation available.

The Jim and Tonic show

Wed, 02 May 2018 13:16:53 +0000

Since starting at Microsoft as a Senior Cloud Developer Advocate I’ve been hugely privileged surrounded on a daily basis by some of the most awesome people across a wide range of developer communities - the rest of the Cloud Developer Advocacy team. These are engineers who constantly work to make the world amazing for developers across a wide range of technologies and disciplines.

To help me get to know the rest of the team, I’ve decided to record a podcast. This is a regular show where I chat to different members of the team to get to know who they are, how they got to where they are now, and just talk about whatever geeky stuff excites them.

It’s called the Jim and Tonic Show.

You can find it at https://www.jimandtonic.show.

Currently two episodes are published:

Asim Hussain

Cecil Phillip

You can also subscribe on iTunes.

Face identification with Azure Cognitive Services Face API

Fri, 02 Mar 2018 20:27:18 +0000

Recently I’ve been playing with a lot AI and seeing how it can be used in mobile apps to enhance the experience offered to the user. Currently I am playing with facial recognition using the Azure Cognitive Services Face API. This is a stupidly powerful API that can do a LOT of different things:

Detect faces in images
Analyze those faces to detect characteristics such as hair color, gender, age
Detect the different points on a face, such as the pupils or lips
Compare two faces and verify if they are from the same person
Group faces by similar facial characteristics
Identify faces from a repository of up to a million images

This is a very powerful set of APIs with a large number of different use cases. For example if you were building a social network you could use the facial identification to automatically tag peoples friends in images. If you were building a ride share system you could use facial verification to ensure the driver is who you expect it to be. For now I’m going to focus on one particular example - identifying faces from a photo in a mobile app.

Face Finder

I’ve built a sample mobile app to show off the facial recognition tools in this API, and you can grab the code from my GitHub repo. This app takes a photo, then finds all the faces in that photo, giving a breakdown of the details of each face.

In the rest of this post I’ll go through you can get signed up for FaceAPI, and how the app works.

Getting started with FaceAPI

From the Azure Portal, select Create a resource, search for “Face”, and select Face from the AI + Machine Learning category. The click Create.

Enter a name for this resource, select your subscription and the location nearest to you. For the pricing tier, there is a free tier called F0 that gives you 30,000 calls per month at a rate of no more than 20 per minute, and you can have one face resource per subscription with this tier. After this there is a paid tier limited to 10 calls per second and you pay per 1,000 calls - at the time of writing this is US$0.25 per 1,000 calls.

Choose a resource group, or create a new one and click Create. Once it has been created, head to it and grab an API key from Resource Management->Keys, and the endpoint from the Overview blade.

Building and running the app

The Face Finder app is pretty complete, all you need to do is update the ApiKeys.cs file with your API key and endpoint. For the FaceApiKey, copy yours and paste it in. For the FaceApiEndpoint, paste the value for the endpoint, removing everything past microsoft.com. For example, for me the endpoint shown in the Azure portal is https://westeurope.api.cognitive.microsoft.com/face/v1.0, so I would set the endpoint to https://westeurope.api.cognitive.microsoft.com.

If you get a Not Found exception, then check your endpoint - this exception is thrown if you don’t remove everything past microsoft.com.

Once you have done this, build and run the app. When it loads, tap the Take photo button and take a picture of one or more faces. The app will then show a list of all the faces detected, describing them using the detected age and gender. Tap on a face in the list to see more details, including if that person is smiling, if they are wearing glasses, what hair, facial hair and makeup they have, and their emotion.

So how does it work

This app is a simple Xamarin.Forms app, with three pages and some view models. The first page, FaceFinderPage.xaml has a button you tap to take a photo, wired up to a command on the FaceFinderViewModel. This uses the Xam.Plugin.Media plugin from James Montemagno to launch the camera and take a picture. This picture is then run through the face API.

The face API is accessed via an SDK from a NuGet package. Currently there are a load of NuGet packages from Microsoft with names containing ProjectOxford - the code name for the various vision cognitive services. These are being replaced with new packages that are called Microsoft.Azure.CognitiveServices.*, and these packages are currently in pre-release. For the face API, I’m using the Microsoft.Azure.CognitiveServices.Vision.Face package, currently available as a pre-release package.

The important class to note here is FaceClient, this wraps a connection to the Azure Face Api and is configured using your API key and endpoint.

Initializing the FaceClient

Before you can use the Face API you have to configure it to use one of your keys and the appropriate endpoint. When constructing an instance of FaceClient you need to pass it credentials, in the form of an instance of ApiKeyServiceClientCredentials which takes one of the API keys assigned to your account as a string in a constructor argument:

var creds = new ApiKeyServiceClientCredentials("<your api key>");

You can then pass this to the constructor of the FaceClient:

var faceClient = new FaceClient(creds);

Finally you set the appropriate endpoint to match the endpoint shown with your keys:

faceClient.Endpoint = "https://westeurope.api.cognitive.microsoft.com";

Detecting faces

Once you have your instance of the FaceClient you can then use that to detect faces using the Face property to access all the different face operations the API supports. The method I’m interested in is the DetectWithStreamAsync method. This takes a stream containing the image, and sends it up to Azure to detect faces. The Async suffix is because it is an async method that you can await (not sure why they’ve added this suffix - they don’t have non-async versions on the API).

var faces = faceClient.Face.DetectInStreamAsync(imageStream);

The imageStream comes from the media plugin. When you use this plugin to take a photo it returns a MediaFile which has a method to get the image as a stream that can be passed to the detect call. This method has some other parameters on it which we’ll look at later.

The results of this call is a list of detected faces - literally a List<DetectedFace>, with one entry per face that was detected in the image. Each DetectedFace in the list contains a set of properties about that face, including coordinates of a rectangle that shows where the face is in the image. The picture below shows a face with this rectangle drawn on top.

Detecting face attributes

So far, so good - we can find where a face is. Now what about more details about the face? This is where the extra parameters on the DetectWithStreamAsync method come in.

DetectWithStreamAsync(Stream image,
 bool? returnFaceId = true,
 bool? returnFaceLandmarks = false,
 IList<FaceAttributeType> returnFaceAttributes = null,
 CancellationToken cancellationToken = default(CancellationToken));

So what do these parameters do:

returnFaceId - set this to true (the default) to return an Id for the face. This Id can then be used for face searching operations - outside the scope of this post!
returnFaceLandmarks - set this to true (default is false) to return the coordinates of the facial landmarks, for example the positions of the pupils, nose, lips etc.
returnFaceAttributes - this is a list of the different face attributes you want returned. There are a lot of these! In the Face Finder app I get them all, and they are:
- Age - a guess at the age of the face. Seeing as it predicted me at 9 years older than I am in the image above it’s either buggy, or (more likely) I need more sleep and to look after myself!
- Gender - a guess at the presented gender. Just limited to male or female.
- HeadPose - what position the head is in, pitch, roll and yaw.
- Smile - the percentage certainty that the face is smiling.
- FacialHair - the percentage certainty that the face has a beard, mustache or sideburns.
- Glasses - the type of glasses (if any) the person is wearing, so normal glasses, sun glasses etc.
- Emotion - the percentages that the face is displaying a set of emotions (e.g. anger, happiness, surprise)
- Hair - the percentages certainty that the face has different colored hair. If no hair is detected then this list is empty, otherwise it covers all natural hair colors and ‘other’. This also specifies if the face is bald or if the hair is invisible (such as under a hat or scarf).
- Makeup - The percentage certainty that the face has eye or lip makeup on.
- Occlusion - how much of the face is occluded (such as by a mask, bandana, hair etc.)
- Accessories - any accessories on the face, such as glasses or a hat
- Blur - how blurry the face is.
- Exposure - how well exposed the picture is.
- Noise - how much noise there is in the image.

These landmarks and attributes come back on the DetectedFace instances, and are present for all faces in the image.

What sort of apps can I build with this?

This API is great and can provide a LOT of power, but are they just for mucking around, or can we build real-world apps with them? Well a few examples I can think of without trying too hard are:

Passport photo app. Passports have strict requirements about photos, so you could use this API to ensure the person was looking at the camera (HeadPose), no glasses (Glasses) or make-up (Makeup), and a neutral expression (Emotion). The photo needs to be of good quality (Exposure, Blur, Noise).
Facial blurring. You could detect the face rectangle and blur out faces automatically, for example in apps that make photos public. You could even do it by age to only blur out children if needed.
Adding decoration to faces. Using the landmarks you could add fake glasses, hats, dog noses or other things to a face to make a comedy picture. I’ve even seen an example replacing faces with emojis that match the emotion being shown.
Auto-picture selection. You could take a selection of pictures and have it choose the best one based on quality (Exposure, Blur, Noise) and if the people in it are smiling (Emotion).

Where can I learn more?

We’ve got loads of great content on-line showing all the cool things you ca do with this API, from all different languages. Check them out:

Read the offical docs at docs.microsoft.com
Work through some great tutorials from the Microsoft AI School

Identifying my daughters toys using AI - Part 5, Plugin for on-device models

Mon, 26 Feb 2018 15:22:18 +0000

In the first part of this series I used the Azure Custom Vision service to create an image classifier to allow me to easily identify my daughters cuddly toys. Once created I tested it by uploading an image and seeing what tags the classifier found for the image.

In the second part I accessed this model from a Xamarin app, so that I could use the camera to take a photo to run through the classifier using a NuGet package that talks to the Custom Vision service.

In the third part I showed how to download this model for iOS and run it locally, on device, using CoreML.

In the fourth part I showed how to download this model for Android and run it locally, on device, using Tensorflow.

To run these models locally is a lot of boilerplate code, so I decided to make it easier and create a NuGet package containing a Xamarin plugin that can be used from iOS, Android and Xamarin.Forms apps.

This plugin makes it easy to download and use these models offline from inside your mobile app, using CoreML on iOS or Tensorflow on Android. These models can then be called from a .NET standard library, using something like Xam.Plugins.Media to take photos for classification.

Setup

Available on NuGet: https://www.nuget.org/packages/Xam.Plugins.OnDeviceCustomVision/
Install into your .NET Standard project and iOS and Android client projects.

Platform Support

This requires iOS 11 and up to use models from CoreML. For Android with Tensorflow the minimum supported SDK version is 21.

Usage

Before you can use this API, you need to initialise it with the model file downloaded from CustomVision. Trying to classify an image without calling Init will result in a ImageClassifierException being thrown.

iOS

Download the Core ML model from Custom Vision.

Pre-compiled models

Models can be compiled before beiong used, or compiled on the device. To use a pre-compiled model, compile the downloaded model using:

xcrun coremlcompiler compile <model_file_name>.mlmodel <model_name>.mlmodelc

This will spit out a folder called <model_name>.mlmodelc containing a number of files. Add this entire folder to the Resources folder in your iOS app. Once this has been added, add a call to Init to your app delegate, passing in the name of your compiled model without the extension (i.e. the name of the model folder without mlmodelc) and the type of model downloaded from the custom vision service:

public override bool FinishedLaunching(UIApplication uiApplication, NSDictionary launchOptions)
{
 ...
 CrossImageClassifier.Current.Init("<model_name>", ModelType.General);
 return base.FinishedLaunching(uiApplication, launchOptions);
}

Uncompiled models

Add the downloaded model, called <model_name>.mlmodel, to the Resources folder in your iOS app.Once this has been added, add a call to Init to your app delegate, passing in the name of your model without the extension (i.e. the name of the model folder without mlmodel) and the type of model downloaded from the custom vision service:

public override bool FinishedLaunching(UIApplication uiApplication, NSDictionary launchOptions)
{
 ...
 CrossImageClassifier.Current.Init("<model_name>", ModelType.General);
 return base.FinishedLaunching(uiApplication, launchOptions);
}

The call to Init will attempt to compile the model, throwning a ImageClassifierException if the compile fails.

Android

Download the tensorflow model from Custom Vision. This will be a folder containing two files.

labels.txt
model.pb

Add both these files to the Assets folder in your Android app. Once these are added, add a call to Init to your main activity passing in the name of the model file and the type of model downloaded from the custom vision service:

protected override void OnCreate(Bundle savedInstanceState)
{
 ...
 CrossImageClassifier.Current.Init("model.pb", ModelType.General);
}

Note - the labels file must be present and called labels.txt.

Calling this from your .NET Standard library

To classify an image, call:

var tags = await CrossImageClassifier.Current.ClassifyImage(stream);

Passing in an image as a stream. You can use a library like Xam.Plugins.Media to get an image as a stream from the camera or image library.

This will return a list of ImageClassification instances, one per tag in the model with the probabilty that the image matches that tag. Probabilities are doubles in the range of 0 - 1, with 1 being 100% probability that the image matches the tag. To find the most likely classification use:

tags.OrderByDescending(t => t.Probability)
 .First().Tag;

Using with an IoC container

CrossImageClassifier.Current returns an instance of the IImageClassifier interface, and this can be stored inside your IoC container and injected where required.

Getting the code

If you want to see the code for this, head to https://github.com/jimbobbennett/Xam.Plugins.OnDeviceCustomVision.

Identifying my daughters toys using AI - Part 4, using the models offline on Android

Wed, 24 Jan 2018 17:43:53 +0000

In the third part I showed how to download this model for iOS and run it locally, on device, using CoreML.

In this part we’re going to switch OS and run these models on Android. Like with iOS, Android offers some APIs to run AI models on device, taking advantage of the hardware. Unlike iOS this is not an API that is baked into the OS, instead it is a separate library you can add to your project and run. The upside of this is it has more OS support, working all the way back to API 21.

The library in question is called TensorFlow, and is actually an open source AI library that came from Google that can run on pretty much any platform and is accessible from most languages. When running on device you can think of it as similar to CoreML on iOS, a generic way of running all sorts of machine learning models including the ones generated by the custom vision service.

Running models with TensorFlow

For mobile apps we can use the Android bindings for the TensorFlow library. Like with CoreML these bindings contain a full API for running all sorts of models, as well as an easy API surface that we can use to do image classification - similar to the CoreML Vision APIs.

To download the TensorFlow model, head to the Performance tab in the Custom Vision portal, selected the latest iteration from the list on the left (we’ll cover iterations in a future post), and click the Export link at the top. Select Android (TensorFlow) then click Download. This will download a zip file containing 2 files:

model.pb - This is the actual TensorFlow model
labels.txt - The labels for the tags

Once these files have been downloaded you will need to add them to the Assets folder in your Android app.

Importing the Android TensorFlow bindings

The TensorFlow Android bindings consist of a native library with a Jar binding. To use these from a Xamarin app we need to add Xamarin bindings, and this has been done for us already by Larry O’Brien from Xamarin and is available on GitHub here: https://github.com/lobrien/TensorFlow.Xamarin.Android. This binding is not available on NuGet yet, so you will need to clone this repo and compile it yourself. Once you’ve compiled it you will need to add the resulting TensorFlowXamain.dll to your Android app.

Creating the model

The TensorFlow binding includes a class called TensorFlowInferenceInterface which can be used to easily run image classification models.

using Org.Tensorflow.Contrib.Android;
...
var assets = Android.App.Application.Context.Assets;
var inferenceInterface = new TensorFlowInferenceInterface(assets, "model.pb");

This will then load the model.pb model file from the asset catalog into a model.

Once we have the model we need to feed it some data, run it, then extract and interpret the output.

Feeding the model

Just like with CoreML the model doesn’t understand images as such, instead it needs binary data in the same format - a 227x227 sized array of 32-bit ARGB values. Again this is sort of pretty easy to create from an Android Bitmap, we just need to convert the bitmap to one of the right size and color space. Once it’s in the right color space we need to average out the colors - models work better when the average of all the color values is 0 (neural networks work better when the average of all inputs is 0, thanks to the ever awesome Frank Krueger for teaching me this). Different domains are trained in different ways, so will need different adjustments.

You can see some Java sample code for this at https://github.com/Azure-Samples/cognitive-services-android-customvision-sample, with details on the adjustments you need to make to the image bytes detailed in the ReadMe.

Luckily I’ve done the hard work converting it to Xamarin for you and it’s on GitHub here.

Once you have the binary data, you pass it to the TensorFlow inference interface as named data, called “Placeholder”. This name is required by the models exported from the Custom Vision service:

inferenceInterface.Feed("Placeholder", floatValues, 1, InputSize, InputSize, 3);

The additional parameters provide details on the float buffer - I honestly don’t know what the “1” parameter is for, the next three are the size of the buffer (in our case 227 for both) and the number of floats per pixel in the buffer - 3 for R, G and B.

Running the model

Once the model has been fed, it needs to be run for a list of outputs - models can produce multiple outputs so we need to run it for all the outputs we need. In our case the only output we need is called “loss”:

inferenceInterface.Run(new[] { "loss" });

Getting the outputs

Once the model has run, we can extract the output that we are interested in - in our case the “loss” output. This comes back as an array of floats containing one entry per tag, with the values representing the probability of the image matching that tag as a value between 0 and 1, 1 being 100% probability. We have to pre-create this array before passing it in - so how do we know how big it is, and how do we know which tag value is which?

The answer comes from the labels.txt file that was downloaded along with the model.pb file. This file contains a list of tags, one per line. Add this file to your apps assets, then load it using:

var assets = Android.App.Application.Context.Assets;
using (var sr = new StreamReader(assets.Open("labels.txt")))
{
 var content = sr.ReadToEnd();
 var labels = content.Split('\n')
 .Select(s => s.Trim())
 .Where(s => !string.IsNullOrEmpty(s))
 .ToList();
}

This will give you a list of labels - the file contains an empty line on the end by default, so remember to trim whitespace and remove any empty lines. You can then create a float array of the same size and put the output in there:

var outputs = new float[labels.Count];
inferenceInterface.Fetch("loss", outputs);

The float values map index for index with the label - so if labels[0] was foo and labels[1] was bar, outputs[0] would be the probability of the image being foo, and outputs[1] would be the probability of the image being bar.

TensorFlow is only supported on API 21 and above - so don’t forget to set your minimum supported version to 21 in your application.manifest

You can read more on exporting and using models here. In the next post in this series we’ll look at the plugin NuGet package I’ve created to make it easy to use these models from a cross platform app.

Identifying my daughters toys using AI - Part 3, using the models offline in iOS

Wed, 17 Jan 2018 17:13:02 +0000

The limitation with this NuGet package is that it requires internet access. Not such a huge problem for classifying single images with fast, cheap data contracts, but not so good in countries where data is expensive, or when classifying a live video feed.

Identifying toys is fun, and is a great project I can share with my daughter, but there are a lot or serious uses for this kind of technology, especially in medical imaging.

Imagine an app where you can run your phone over moles on your skin and have it tell you in real-time if you are at risk of skin cancer. You’d want this to run on device for a couple of reasons. Firstly for speed, uploading a lot of images is slow. Secondly, and more importantly, this kind of technology is incredibly useful for third-world countries who have little in the way of education around health issues, or access to medical screening services. These countries also have little or no, or incredibly expensive internet access, especially away from big cities where people are more likely to be lacking medical screening. If you could take a phone on the road with these models running locally you could easily screen whole villages in a short space of time.

This kind of app is not a sci-fi dream, but is a real thing. You can see an example iOS app and download actual trained models for identifying skin cancer at https://github.com/Azure/ai-toolkit-iot-edge/tree/master/Skin%20cancer%20detection.

These models run on device using CoreML on iOS and TensorFlow on Android. In this post we’ll look at CoreML, in the next post we’ll look at TensorFlow.

Running models on CoreML

CoreML was introduced in iOS 11 and macOS High Sierra, and is a set of APIs for running models on device instead of up in the cloud. The models used are compressed and optimized to run on device GPUs, and they run pretty quickly. CoreML can run all sorts of models, and has a specialized API designed for running image classification models that we can take advantage of.

To download the CoreML model, head to the Performance tab in the Custom Vision portal, selected the latest iteration from the list on the left (we’ll cover iterations in a future post), and click the Export link at the top. Select iOS 11 (CoreML) then click Download. This will download a .mlmodel file. Before you can use this model, you will need to compile it - either in advance or at run time.

To compile in advance on a Mac using the following:

xcrun coremlcompiler compile <model_name>.mlmodel <compiled_model_name>.mlmodelc

The model_name will be a GUID, so set the compiled_model_name to be something more user friendly. Once this is compiled you will have a folder called compiled_model_name.mlmodelc containing some model files. You will need to add this entire folder to your Resources folder in your iOS app, so drag this into Visual Studio.

You can also compile it at runtime. This is useful especially if you want to download new models without updating your app. If you want to compile at runtime, just add the <model_name>.mlmodel file to your resources folder, renaming the model to something easier to read than the default Guid.

To run CoreML models you need three components:

The model
A request for the model with a callback
A request handler to issue the request

The model

Once we have the model as a resource, we need to load it into a CoreML model, then convert it to a vision model:

using CoreML;
using Vision;
...
var assetPath = GetModel();
var mlModel = MLModel.Create(assetPath, out NSError err);
var model = VNCoreMLModel.FromMLModel(mlModel, out err);

The GetModel method varies depending on if the model is compiled or not. For compiled models use:

private NSUrl GetModel()
{
 return NSBundle.MainBundle.GetUrlForResource("<compiled_model_name>", "mlmodelc")
}

To compile your model on device use:

private NSUrl GetModel()
{
 var url = NSBundle.MainBundle.GetUrlForResource("<model_name>", "mlmodel")
 return MLModel.CompileModel(url, our NSError err);
}

In a real app you should check the NSError out variables for any errors

This code creates a MLModel from a URL to a resource, then converts this to a VNCoreMLModel - a ‘Vision’ CoreML model for image classification constructed using the custom vision model. The Vision namespace also contains some built in models for bar code recognition and facial recognition, amongst others.

If you want more information on this class, the docs for VNCoreMLModel are here.

The request

The CoreML request wraps our model and provides a callback. Running these models is asynchronous - once you start it, it will run on the GPU in a background thread and invoke the callback once done.

The code to create the request is:

var request = new VNCoreMLRequest(_model, (response, e) => {});

The second parameter is the callback, with the response parameter containing the results from the model, and the e parameter being an NSError instance in case of an error.

If you want more information on this class, the docs for VNCoreMLRequest are here.

The request handler

Although this model is for image classification, it actually doesn’t understand what an image is - it works on raw binary data only. The custom vision models expects this binary data to be in a specific format, namely a 227x227 CoreVision pixel buffer using 32-bit ARGB, whatever that is! Luckily it’s easy to create one of these using code I found on one of the Xamarin iOS CoreML samples. Follow that link for an extension method that converts from a UIImage to a pixel buffer.

Once we have the pixel buffer, we create a request handler for our buffer:

var buffer = source.ToCVPixelBuffer(_targetImageSize);
var requestHandler = new VNImageRequestHandler(buffer, new NSDictionary());

We can then invoke the request handler with our request:

requestHandler.Perform(new[] { request }, out NSError error);

This handler can be used to run multiple requests at once on the same pixel buffer, just pass all the requests you want to run as items in the array in the first parameter. This is useful if you wanted to screen for multiple conditions in a medical imaging app, with multiple models for each condition run against the same image.

If you want more information on this class, the docs for VNImageRequestHandler are here.

Handling the response

Once the model has been run, it will call the callback you passed to the request, passing in the response (or an error if it failed). This response has a method on it, GetResults<T>() that returns an array of results of whatever type the model spits out. For vision models, the results are of type VNClassificationObservation, so inside our callback we would do:

var results = response.GetResults<VNClassificationObservation>();

VNClassificationObservation has two fields that we are interested in:

Identifier - this is the tag we set in our model
Confidence - this is the probability that our image matches our tag, from 0 to 1 with 1 being 100% confidence.

These values match up to the Tag and Probability fields from the ImageTagPredictionModel class in the Custom Vision NuGet package. You can read more on the VNClassificationObservation class here.

For an example of this code in action, check out my Xamarin.PlugIn.OnDeviceCustomVision NuGet package. This wraps the code above, along with an Android implementation using TensorFlow (which we’ll look at in the next post) in an easy to use Xamarin Plugin.

CoreML is only supported on iOS 11 and above - so don’t forget to set your deployment target to iOS 11 in your info.plist

You can read more on exporting and using models here. In the next post in this series we’ll see how to run models on device on Android using TensorFlow.

Identifying my daughters toys using AI - Part 2, using the model

Tue, 02 Jan 2018 17:43:37 +0000

In the first part of this post I used the Azure Custom Vision service to create an image classifier to allow me to easily identify my daughters cuddly toys. Once created I tested it by uploading an image and seeing what tags the classifier found for the image.

Although this works, it’s a long winded way of using the model. Ideally I’d want it in a mobile app (after all, I am a bit of a Xamarin fan) so in this post we’ll look at calling the classifier from a Xamarin app.

The first thing to do is to create a new blank Xamarin Forms app, I’m going to call mine Toy Identifier. This app will be pretty simple to start with - one screen with a big button to launch the camera, then after a photo is take I’ll use text to speech to say the name of the toy.

Like a lot of Xamarin apps, the bulk of this will just be some of James Montemagno’s Plugins joined together, so add the following packages to all the projects (core, iOS and Android):

Xam.Plugin.Media
Xam.Plugins.TextToSpeech

The Media plugin has some permissions you need to set, so follow the instructions at https://github.com/jamesmontemagno/MediaPlugin to configure this.

We’ll also need the Custom Vision client side SDKs, so add:

Microsoft.Cognitive.CustomVision.Prediction
Microsoft.Rest.ClientRuntime

as well to all your projects.

You can find all the source code for this app on GitHub at https://github.com/jimbobbennett/ToyIdentifier, so grab the code now.

The UI is simple - a button and a label. Tap the button and the camera appears, take a photo and the label shows the name of the toy. I am using MVVM, but not bothering with a framework at this point as the app is too simple, so I’ve got a view model that is created inside my XAML which has all the fun logic.

If you open ToyIdentifierViewModel.cs you’ll see how simple it is to take photos and upload them to the custom vision service. Taking a photo is easy thanks to James Montemagno’s Media plugin:

var options = new StoreCameraMediaOptions { PhotoSize = PhotoSize.Medium };
var file = await CrossMedia.Current.TakePhotoAsync(options);

I’m using a medium sized photo - this returns an image which is 50% of the size of the one that comes out of the camera. This is fine as we don’t need a high res image for recognition, and the serviced apartment I’m staying in at the moment has crappy internet so the smaller the better.

The TakePhotoAsync call returns a MediaFile object, with a method on it to return the file as a stream, and we can pass this stream to a call to the custom vision SDK.

To use the custom vision SDK we need to start by creating a PredictionEndpoint. Uploading an image to get the percentage chance of it matching the tags is called prediction.

private PredictionEndpoint _endpoint = new PredictionEndpoint { ApiKey = ApiKeys.PredictionKey };

The PredictionKey I’m passing in comes from a constants file - you’ll need to update this file with your own API key. You can find this by going to the settings on your project in the custom vision portal and getting the value of the Prediction Key. There is also a ProjectId in that same file you need to set, again this comes from the settings page.

Using the prediction endpoint is easy - there is a PredictImage method on this class, and this takes the project id as a Guid, and a stream containing the image to use for the prediction.

_endpoint.PredictImage(ApiKeys.ProjectId, stream)

This call uploads the image to the custom vision service and returns a model containing information about the custom vision project, and a list of predictions - essentially a list of all the tags the a numerical probability that the image matches the tag in the range of 0 to 1, the higher the number, the more likely the match.

Once we have these predictions we can sort them by probability then get the highest. We then need to add a threshold - after all if we match all tags with a probability < 0.01 then it’s unlikely the image is one of our tags. In my code I’ve got a threshold of 0.5 to start with, but I might increase this after playing some more.

_endpoint.PredictImage(ApiKeys.ProjectId, stream)
.Predictions
.OrderByDescending(p => p.Probability)
.FirstOrDefault(p => p.Probability > 0.5);

Once I have the best match I update the label to say hello to the particular toy, then use the text to speech plugin from James Montemagno to actually say hello to the toy. If there are no matches the label is changed to say “I don’t know who that is”.

If you want to see more on the SDK, it is on GitHub at https://github.com/Microsoft/Cognitive-CustomVision-Windows. At the time of writing the code in there is a bit out of date and doesn’t match the NuGet package, so check out this pull request to find the latest code.

Check out this short demo:

Want to read more on how to get started with the Custom Vision APIs? Check out the docs here.

In the next post we’ll look at exporting these models to run on an iOS device.

Hiding API keys from Git

Thu, 28 Dec 2017 19:35:50 +0000

I’ve been working on a Xamarin app using Azure Cognitive Services to do image recognition, and one of the stumbling blocks I’ve faced is what to do with my API keys. I want to make the app open source as an example of how to use these services, but don’t want to check my API keys into Git to be available to all - after all, bad things can happen.

I hit up twitter, and got a really awesome solution from Bart Lannoeye:

You can add the file with *your key here* then change locally and git update-index --assume-unchanged .\pathtofile. Shows file as unchanged => can't commit by accident. #protip We do the same for Kliva.
— Bart Lannoeye (@bartlannoeye) December 28, 2017

This is exactly what I did, and it works perfectly. I’ve created a static class called ApiKeys which contains all my keys using “Your Key Here” values:

public static class ApiKeys
{
public static string PredictionKey = "<Your API Key>";
public static Guid ProjectId = Guid.Parse("<Your Project GUID>");
}

I then added this to Git and commited. After my commit I ran:

git update-index --assume-unchanged ./ApiKeys.cs

Done. I can then change the values to my actual API keys and Git doesn’t see the change.

Obviously if I need to add any more keys to this file I’d have to revert this change, remove all keys, add the new one with a “your key” type value, commit, re-run the update-index and put the keys back. A bit of work, but at least no worries about anyone abusing my API limits!

Thanks Bart!

Update - how to tell users what they need to do with this code

I’ve just had another great suggestion from Brandon Minnick, a fellow CDA here at Microsoft. He suggests adding a #error to the keys file so that when someone grabs the code and builds it they get an error telling them what to do, rather than a crash when the app is run:

public static class ApiKeys
{
# error You need to set up your API keys.
// Start by registering for an account at https://customvision.ai
// Then create a new project.
// From the settings tab, find:
// Prediction Key
// Project Id
// and update the values below
public static string PredictionKey = "<Your Prediction Key>";
public static Guid ProjectId = Guid.Parse("<Your Project GUID>");
}

Update 2 - how to fix it if you forget and add your keys

Another great tip from Brandon Minnick is the BFG Repo-CLeaner. If you accidentally checked in some API keys, this tool can remove them from the Git history.

Obviously it could be too late by the time you realize, so if you check any API keys in to a public repo you MUST regenerate them as there are bots that can GitHub for API keys. But this is good for a private repo that you are planning to make public or accidentally add personal keys to.

Identifying my daughters toys using AI - Part 1, creating the model

Wed, 27 Dec 2017 14:30:09 +0000

Prefer to watch instead of read? - https://youtu.be/3sjInlJcpBs

It’s currently just after Christmas, and my 4 year old daughter has received a fair few cuddly toys from Santa. These bring her total collection up to way too many, and like most parents I have a hard time remembering all their names, especially the least favorite ones that rarely come out.

I’ve thought what would be really useful would be a mobile app that allows me to easily identify each toy. Luckily such an app is really, really easy to build using the power of AI.

One of the cool things you can do with neural networks is image classification. You set up the network with a training set of images and assign tags to each one. Once trained you can pump any image into the network and it will return a set of percentages that show how much it thinks the image matches all the tags it’s learned about. For my app idea I can upload images of each toy, and tag those image with the name of the toy.

Normally training these networks takes a LOT of images - think hundreds or thousands. This could be problematic as I really cannot be bothered to take thousands of pictures of my daughters toys, not when there’s Christmas leftovers to be eaten.

As it turns out, we actually don’t need thousands of images. Microsoft have created a custom vision service as part of their Azure Cognitive Services that can do it with 5. Yup, only 5. Their neural network has already been partially trained for image recognition using some very clever technology that I have absolutely no chance of understanding, so you just upload a small set of pictures, assign a tag and away you go. 5 is a minimum, they do recommend at least 30 for better accuracy, but you can start with 5.

You can access this at https://customvision.ai. Head there, sign up or log in with a Microsoft account. This service is currently in preview, so is available for free for now.

Once you have logged in, click ‘New Project’ to create a new image classifier project.

A side bar will pop out where you can enter a name for this project, a description and select a domain. These domains are part of why this classifier can run with only a few images per tag in its training set - it’s configured to be an image classifier, and they have a number of pre-configured networks to specialize in certain things. For example they have a version which is better when it comes to food, another that is better for landscapes, another that is better for filtering out adult content.

You can read more on the different domains in the developer docs.

After entering a name and description, select a domain. You’ll notice there are two versions of some domains - normal and compact. The compact ones are smaller models that can be downloaded as either a CoreML file or a TensorFlow file. These are model files you can import into a mobile app to run on the device. Create a General (compact) model for now so we can download it later on (I’ll cover this in more detail in a later post).

Once your project is created you need to train it, so select the Training Images tab and upload a set of images that match one of the tags you’re interested in. In may case, I’m starting by uploading 7 images of a cuddly duck who goes by the name of ‘Baby Duck’.

I then repeat the process with her remaining toys. Once I’ve uploaded the photos of a few toys my model is ready to train.

Training is easy - click the green Train button at the top of the page.

This kicks of the model training, and it will take a few seconds. More if you have a lot of images. Once done you have a working model, so it’s time to try it out. Click the Quick Test button and upload an image. After a few seconds your image will be classified - against each tag you will see a percentage chance that the image matches the tag. I tested mine using Miss 4’s Porg and it recognized it with a 99.9% probability.

Every time your model is run against an image, the image is stored along with it’s scores so that you can use it to validate and re-train your model. These images appear in the Predictions tab. You can select each image and add the correct tag to help improve the model - either by clicking on them one by one or by ticking them all and selecting Tag Images. Once you’ve added tags you will need to retrain the model using the Train button.

We now have a trained model. In the next post, we’ll look at using it from a Xamarin app.

Want to read more on how to get started with the Custom Vision APIs? Check out the docs here.

Getting a users Facebook profile after Authenticating your Xamarin app with Azure

Fri, 17 Nov 2017 23:54:08 +0000

I’ve recently been looking at the authentication offered by Azure for use in a mobile app, specifically using social auth providers like Facebook to allow a user to sign up for my app. The auth setup is incredibly easy, with Azure taking care of a lot of the hard stuff. There are also loads of great docs on how to get it set up - including the Microsoft docs available here - https://docs.microsoft.com/en-us/azure/app-service-mobile/app-service-mobile-xamarin-forms-get-started-users.

The whole login flow is very well documented, so I won’t repeat what’s in the previous link. Just follow the instructions in that link and you can use your Xamarin app to authenticate with Facebook and get back a logged in user.

What does seem to be missing from the docs though, is what happens next. When you log in using the MobileServiceClient.LoginAsync method you get back an access token, and that’s it. So what is this access token, and how can it be used to access the users personal information, such as Facebook photo or friends? That’s what we are going to look at in this post.

When you make a call to LoginAsync, the SDK opens a web view pointing to your app service, which in turn redirects to the Facebook login. When you log in, the web view redirects to your app service, which redirects back to your app. Along the way, the Azure app service picks up an access token from Facebook, which it keeps hold of, and returns you an access token from Azure.

Your app doesn’t get the Facebook access token, just an Azure one. Instead your app service keeps hold of the Facebook access token. This I imagine is intentional - your app can authenticate with multiple providers at the same time, for example linking to both a Facebook and Twitter account. Instead of returning access tokens to your mobile app for all of these providers, it returns just one, and this can be used to extract all the information you require via an auth service built into the Azure app service.

This auth service is available from the https://<my azure website>/.auth/me end point on your app service, passing the access token as a X-ZUMO-AUTH header. This returns a JSON document containing information from all social auth providers that the user has signed in with, including the access tokens, as well as some basic personal information such as first and last name. The Facebook access token that comes back can then be used with the Facebook Graph API to download whatever details we’ve granted to the app, such as our profile pic or friends list.

From my mobile app I want to download the users Facebook profile, but I don’t want to make multiple server calls to do it. From my app I could call the /.auth/me endpoint, get the access token, then make another call to the Facebook graph, but it might be better to do it in one call, handling all the steps server side (including the logic around which social provider to use if more than one was provided). By doing this server side I can also cache information in my app service, maybe for later processing. The easiest way to do this is via an Azure Function.

If you’ve been living under a rock for the past few years you may have missed out on the rise of serverless, but I imagine most of you are familiar with the concept. Azure functions are simple functions that are run in response to a trigger, such as a web request, a timer or a message on a queue. These run on someone else’s hardware (hence the same serverless - you don’t have to provision a server to run them on, someone else does it for you), and you only pay for the CPU/RAM usage, not a monthly cost for hardware. This means they are cheap - potentially millions of calls for less than a cup of coffee. They are also infinitely scalable, if your function gets hit a bazillion times it will scale up automatically. If you want to learn more, check out this Channel9 video from Jeremy Likness, a Cloud Developer Advocate at Microsoft.

We can use a simple function to do all our server side processing for us. Log into the Azure portal and create a new Function App. Fill in all the normal details, making the name something simple to remember such as <my app service name>_functions. The hosting plan is a new option, specific to functions. You can choose to link it to an existing app service so that it will only use those resources (and not end up costing any more than the app service rate), or a consumption plan where you pay per usage - with a very generous free tier including being able to run a million function calls before you start paying.

Fill in all the details and click Create. Wait a couple of minutes for your function to be created then head to it. Select the functions node in the tree under your functions app, then select + New Function. From here you can choose from a set of function templates with different triggers and languages. We want a function that uses an Http trigger so that we can call it from our mobile app, and I’m using C# but you can use F# or Javascript if you prefer. Select HttpTrigger - C#, give your function a name such as GetUserDetails, set the authorization level to anonymous (I’ll look at security more in a later blog post), then click Create.

When created you will see your new function, which is essentially a C# script file that you can edit in your browser, with one method in it - public static async Task<HttpResponseMessage> Run(HttpRequestMessage req, TraceWriter log). This method is passed the Http request used to make the call to your function, and returns an Http response with all the usual response stuff, like status code and any content. The function has a sample implementation that looks for a name field in a JSON body and returns a hello message to that name - the canonical Azure functions ‘Hello World’. If you click the Run button, a pane will slide out from the right with a ‘Test’ tab that can be used to test the function. You’ll see the response from the call in the ‘Output’ box, so run this function to see it in action, changing the name in the request body to see how it all works. The browser experience is limited though, so you can’t debug your functions, just run them (but you can write to the log parameter to see output in the log box on the bottom).

The first thing our function needs to do is to call the auth endpoint to download the user details including access tokens for our social providers. To do this, it needs the access token from our mobile app, so we’ll assume this will be passed as an Http header called ACCESS_TOKEN. Delete the code inside the function, and start by adding the line below to read the access token from the headers:

var accessToken = req.Headers.GetValues("ACCESS_TOKEN").FirstOrDefault();

Now we have the token, we need to pass it to our auth end point using the standard C# HttpClient. Add the code below, changing <my site> to the address of your Azure app service (this is your app service used for authentication, NOT your Azure function service):

var client = new HttpClient();
client.DefaultRequestHeaders.Add("X-ZUMO-AUTH", accessToken);
var meResponse = await client.GetAsync("https://<my site>/.auth/me");
var content = await meResponse.Content.ReadAsStringAsync();

This code creates a new HttpClient, adds a X-ZUMO-AUTH header using the access token from the headers, and downloads the content. To see this in action, run your Xamarin mobile app through a debugger, log in using facebook, then copy the value of the MobileServiceAuthenticationToken property on the MobileServiceUser returned by the call to LoginAsync. From the Azure function, go to the test tab, set the Http method to ‘Get’, and add a new ACCESS_TOKEN header with the value set to the MobileServiceAuthenticationToken property. Finally add some code to the end of the function to return the auth response:

var response = req.CreateResponse(HttpStatusCode.OK);
response.Content = new StringContent(content, System.Text.Encoding.UTF8, "application/json");
return response;

If you now run your function, you will see the response from the auth endpoint in the output window as JSON. If you tidy this up, such as using the Prettify JSON extension in VS Code you will see it is an array of objects that contain information for each social auth provider the user is authenticated against, with fields such as the provider_name, a list of user_claims, which is assorted data about the user relevant to the provider, such as name, gender and location. It also contains a field called access_token, which is the provider specific access token that we can use to access the Facebook graph API.

We can get this field from the JSON using our old friend Json.Net. Azure functions have a few NuGet packages that are always available, and one of these is Json.Net. To use it we have to start by referencing the assembly, then add a using directive to the top of the function:

# r "Newtonsoft.Json"
using Newtonsoft.Json.Linq;

We can then use it to parse out the Facebook access token:

var fbAccessToken = JArray.Parse(content)[0]["access_token"].ToString();

If you want to test this out, change the content of the response to show the fbAccessToken instead of content and run the function. You’ll then see a nice long access token string in the output.

Once we have this access token it can be used to query the Facebook Graph API. You can read the docs on the Graph API here and try it out using their Graph API explorer.

Essentially it is an API that you make GET requests to, passing a query string defining the fields you are interested in, and using the access token for bearer authorization. Create a new HttpClient and set the bearer authorization using the Authorization header and a value of Bearer <facebook_access_token> as shown below:

var graphClient = new HttpClient();
graphClient.DefaultRequestHeaders.Add("Authorization", $"Bearer {fbAccessToken}");

Then we can make a call to the Graph API, in this case loading the users first name, last name and profile picture. The fields we want are passed to the graph API call as a comma-separate list passed as a query parameter:

var graphResponse = await graphClient.GetAsync("https://graph.facebook.com/v2.11/me?fields=first_name,last_name,picture");
var graphContent = await graphResponse.Content.ReadAsStringAsync();

Finally change the response to return the graph content:

response.Content = new StringContent(graphContent, System.Text.Encoding.UTF8, "application/json");

Now if you run this, you will see the output from the Facebook Graph API, providing the URL of the users picture, their first name, last name and unique Facebook Id. A sanitized version is shown below:

{
 "picture": {
 "data": {
 "height": 50,
 "is_silhouette": false,
 "url": "<url of my facebook profile pic>",
 "width": 50
 }
 },
 "first_name": "Jim",
 "last_name": "Bennett",
 "id": "<my id>"
}

The basic flow is:

Now we have our function, we can call this from our Xamarin mobile app as soon as the user is logged in. Once we have the user object, we can call our Azure function as if it was any other REST API using the code below:

var client = new HttpClient();
client.DefaultRequestHeaders.Add("ACCESS_TOKEN", user.MobileServiceAuthenticationToken);
var response = await client.GetAsync("https://<my function app>/api/GetUserDetails");
var content = await response.Content.ReadAsStringAsync();

We pass the MobileServiceAuthenticationToken property from the user object as an Http header - this is the access token from our Azure app service. We then call the new function which is at https://<my function app>/api/GetUserDetails. The URL of your function app is <the name you gave it>.azurewebsites.net, unless you have decided to use a custom domain, and you can find this value by clicking on your function app in the tree in the Azure portal and looking at the details on the right.

The content that comes back is the JSON from Facebook, and you can convert this into useful data by building a simple class to represent the data and deserializing the JSON into it using Json.Net. The classes to use to deserialize are:

public class FacebookPictureData
{
public string Url { get; set; }
}
public class FacebookPicture
{
public FacebookPictureData Data { get; set; }
}
public class FacebookDetails
{
[JsonProperty("first_name")]
public string FirstName { get; set; }
[JsonProperty("last_name")]
public string LastName { get; set; }
public FacebookPicture Picture { get; set; }
}

You can then deserialize the content from the function call using:

var facebookData = JsonConvert.DeserializeObject<FacebookDetails>(content);

Done - you now have the users first name, last name and a URL of their public Facebook picture that you can use as an image source for a Xamarin.Forms Image control. If you want more data from Facebook just add more fields to the Graph API call an the details class on your mobile app.

Creating Visual Studio project and solution templates - Part 3, VS for Mac extension

Wed, 08 Nov 2017 07:06:26 +0000

In the first part of this set of posts I looked at creating a dotnet new project template, and in the second part I showed how you could easily add this to a Visual Studio for Windows extension. Lets now look at adding it to a Visual Studio for Mac extension, so that our template is available everywhere.

It’s really easy to create extensions for VS for Mac for new project types. In a previous blog post I discussed the old way of doing it which, like for VS on Windows, involved a load of files that couldn’t be compiled as they had replacement tokens, and no easy way to test the output projects without debugging the extension.

This all changes now. Instead of adding all the project files, you just need to add the NuGet package created for dotnet new with a build action of Add-In File, then add an entry into the Manifest.addin.xml file to make it available:

<Extension path="/MonoDevelop/Ide/Templates">
<Template
id="MvvmCrossNativeSinglePage.CSharp"
_overrideDescription="Creates a new single page native MvvmCross app."
path="Templates/MvvmCross.Templates.CSharp.Native.SinglePage.iOS.Android.nupkg"
category="other/net/mvvmcross"
icon="res:MVVMCross.XSAddIn.Icons.MvvmCross.png"/>
</Extension>

This manifest entry provides a description to show in the File->New dialog, the category in that dialog to put the template into, an icon and the path in the addin to the NuGet package. You can add as many NuGet packages as you want, with one Template entry per NuGet package. And that’s it - no need to add the project files, xpt.xml files or anything hard like that. You can see an example of this with my MvvmCross templates on my GitHub.

Creating Visual Studio project and solution templates - Part 2, VS for Windows extension

Mon, 06 Nov 2017 23:42:10 +0000

In the first part of this set of posts I looked at creating a dotnet new project template. These are great if you like the cli, but if, like me, you’d rather be able to do File->New then dotnet new is not much use. Instead you need a Visual Studio extension that provides a new project or solution type to the IDE. The good news is that you can easily take what you’ve built for your dotnet new templates and create extensions for Visual Studio for both Windows and Mac with not much extra work.

In this post I’ll look at VS 2017 for Windows, in the next post I’ll cover VS for Mac. Although I’ll look is not really correct - instead I’ll be lazy and link to someone else’s content ;op.

There is an extension for VS for Windows called Sidewaffle Creator from Sayed I. Hashimi that provides all the help you need. You load up the project that you generated your NuGet package from into VS, then add a new Template Pack Template project from the SideWaffle extension - this new project is your VSIX.

You can see all this in detail in this YouTube video from Sayed:

Introducing Visual Studio Mobile Center

Sat, 04 Nov 2017 06:52:09 +0000

In this article, excerpted from chapter 15 of Xamarin in Action, we’ll be looking at using Visual Studio Mobile Center’ mobile-optimized lifecycle to build, test, deploy, and monitor applications.

Save 37% off Xamarin in Action with code fccbennett at manning.com.

Introducing Visual Studio Mobile Center

Visual Studio Mobile Center is billed as ‘Mission Control for apps’ and provides a one-stop shop to managing the mobile-optimized developer lifecycle in your mobile apps, covering testing, building, deployment and monitoring.

Build - Mobile Center has tools to take your code and build it either on demand, or when you push to source code control.
Test - Mobile Center integrates with Test Cloud, which allows you to take the UI tests and run them on real devices in the cloud (in this case the ‘cloud’ is a warehouse in Denmark with racks and racks of phones and tablets).
Monitor - Using Mobile Center, you can get analytics around the users who’re using your apps, and crash reports that upload automatically if your app crashes for your users.
Distribute - From Mobile Center you can distribute your builds to beta testers, including adding notifications inside your app when an update is available. You can also distribute to the Google Play and Apple App Stores.

This is slightly out of order in terms of the lifecycle, but makes sense to developers—we should set up the build before we can test it and we should add analytics before we distribute to see the results as soon as possible.

You can access Mobile Center at https://mobile.azure.com. At the time of writing, it’s available as a public preview and is free for light usage. You can log in using either a Microsoft account (you probably set one of these up when installing Xamarin), or by connecting it to your GitHub, Facebook, or Google account.

_Figure 1. Visual Studio Mobile Center: Mission Control for apps_

Once you’ve connected to Mobile Center, it’s worth setting up a Mobile Center password associated with the email address used by the service you connected with. This allows you to log in from the command line later, which is needed to run UI tests. To do this, click on the cog icon next to your name to see the user settings, go to the password tab and click the ‘Send set password email’ button. You’ll receive an email with a link to follow to set a password; click this link and set a password.

NOTE: Mobile Center is a continuously evolving service At the time of writing, Mobile Center is in preview, and it’s being actively worked on. Hopefully, the implementation shouldn’t change too much, the concepts certainly should not.

Mobile Center is based around the concept of ‘Apps’, and apps can belong to users or organizations.

Apps

Mobile Center is built around the concept of ‘Apps’, with each app being a single, releasable app for a single platform. Imagine that we are developing an app called Countr (if you would like to follow along, you can fork the source code from https://github.com/jimbobbennett/Countr). Although we may think of Countr as a single app available on iOS and Android, from the perspective of Mobile Center it’s two apps, one Android, one iOS.

TIP: Mobile Center covers more than Microsoft products As part of the new ‘Any developer, any platform,’ Microsoft has made sure that Mobile Center works for all mobile developers, not only Xamarin developers. They support native iOS and Android apps built using Objective-C, Swift and Java, Xamarin, Windows UWP and React Native—and they supported native iOS and Android before Xamarin. This means that if you work with a mixture of app technologies you can still use Mobile Center to manage everything.

For each app Mobile Center provides a number of services:

Build - Mobile Center can connect to a source code repository (currently GitHub, VSTS and BitBucket are supported), and for each branch in your repo you can create a build to compile your solution or an individual project using any available build configuration, either on demand or whenever the code is updated in the repo, when you sign your build using an Android keystore or iOS provisioning profile, or even launch it on a real device using Test Cloud as a sanity check that your app runs and doesn’t crash on startup.
Test - Xamarin has a huge datacenter in Denmark with thousands of real devices that you can run your app on using automated UI testing. Mobile Center can be used to configure, start and see the results of these test runs.
Distribute - Mobile Center can distribute your app as soon as it’s built, to beta testers or to the Google Play or Apple App stores. Beta test users get an email with a link to download your app to test it, and can even get notifications inside their apps that an update is available.
Crashes - Most apps crash at some point and Mobile Center has an SDK you can add to your apps that can track crashes. If your app crashes, the Mobile Center SDK tracks this, and the next time your app is run and is online it uploads the crash details, including the stack trace, to Mobile Center.
Analytics - It’s always good to know how your users are using your app to track pain points, see what features are popular or aren’t being used to know which areas to work on. It also helps to see user demographics—if your app is in English, but it’s popular in Italy, you might want to offer it in Italian. The Mobile Center SDK can help to track this kind of information.

Users and Organizations

When you log into Mobile Center, you’re logged in as a user and you can have apps assigned to you as an individual. Users can also be part of an organization—a named group of users with different permissions. You can be an ‘admin’ user and have full control over the organization, or a ‘collaborator’ who creates and manages apps inside the organization. You can read more on organizations at https://docs.microsoft.com/en-us/mobile-center/dashboard/creating-andmanaging-organizations.

API

Mobile Center was written to be API first—the development team at Microsoft created public APIs for managing your apps, and the web portal was written to use these APIs. This means that anything you can do in the Mobile Center web portal, you can also do using a public REST API. You can easily integrate Mobile Center into any existing build or CI tools that you already use. For example, if you already have an automated build and release process using a tool, like Jenkins, you can replace the steps for building your app with calls to Mobile Center and use that to run your builds. API documentation is available at https://docs.microsoft.com/en-us/mobile-center/api-docs/.

CLI

Mobile Center also has a command line interface that you can use to do everything Mobile Center can do from a command line. The Mobile Center docs at https://docs.microsoft.com/en-us/mobilecenter/cli/index have more information on the CLI, covering the huge range of features it offers. We’ll look at getting this set up and using it later in this article to run automated tests.

Getting Help

Mobile Center has excellent documentation available by clicking the question mark icon in the bottom of the menu, or at https://docs.microsoft.com/en-us/mobile-center/. You can also interact directly with the Mobile Center team at Microsoft if you get stuck, find bugs or have feature suggestions by clicking the blue and white speech bubble button. From here you can start conversations with the team, see their responses and see messages about new features.

Setting up Builds

Mobile Center apps are based around code for a mobile app from a source control repository, and before we can set up a build we need to put the source code for our app into a repository that Mobile Center can access. At the time of writing Mobile Center supports three providers: Git repos in GitHub and BitBucket; and Git or TFS repos in Visual Studio Team Services. Source code control is outside the scope of this article, and if Git’s new to you I’d recommend Learn Git in a Month of Lunches by Rick Umali. For the rest of this article you’ll need to have a basic understanding including being aware of branches, commits, and pushes.

Let’s have a look at setting up Countr, our fictitious app that we mentioned earlier. You’ll need to add your Countr code to one of the three source code providers to get started. Each one has comprehensive documentation to get you started. I personally use GitHub as you can use a GitHub account to log into Mobile Center, and this automatically gives Mobile Center access to your repos, but use what you feel most comfortable with.

Creating your first Mobile Center app

Once your code is in source control, start by creating an app in Mobile Center. We’ll begin by creating an Android app, and then the iOS version. From the Mobile Center landing page, click the ‘Add new app’ button on the top right.

_Figure 2. Setting up our app in Mobile Center_

A panel slides out on the right where you can enter details for your app. Set the name as Countr - Android, add a description, set the OS to Android and the Platform to Xamarin, then click Add new app. This creates the app and shows a ‘Getting started’ page with details on how to set up the Mobile Center SDK for crash reporting and analytics.

_Figure 3. To create a build you first need to connect Mobile Center to your source code provider_

The next step is to configure a build, and to do this we need to connect Mobile Center to our source code provider. Click the ‘Build’ tab on the menu on the right, select your source code provider of choice and follow the instructions to connect it to Mobile Center. Once connected, choose the repo that you put the source code for Countr into.

Configuring the Android build

Once you’ve set up your repo, the build tab shows all available branches in that repo, and if you set up a new Git repo you’ll probably only see one branch called master. Click on the master branch, and then click ‘Set up branch’. A panel will slide out with options to configure your branch.

_Figure 4. Mobile Center can build your branch, as well as sign it, test it and set it up for distribution_

This configuration is divided into four sections:

Build app - In this section you can configure how your app should be built.
Project - The first option is the project to build - your solution could contain multiple Android apps, and here you can select the one to build. Our solution only has one Android app; Counter.Droid should be selected.
Configuration - From here you can set the build configuration to compile with—either Debug or Release (or any others if you’ve more set up). We’ll need a release build for running UI tests later; set this to Release.
Build frequency - The build frequency can be set to every push to this branch or on demand. Leave this as Build this branch on every push; every time you push one or more commits to GitHub, BitBucket or VSTS the app is rebuilt.
Sign builds - Mobile Center can sign your builds using your Android keystore. Turn on ‘Sign builds,’ upload your keystore from the safe place where you keep it, enter the alias you used when creating it into the KEY_ALIAS, then enter your keystore password into both the KEYSTORE_PASSWORD and KEY_PASSWORD fields (when you set up a keystore using Xamarin it uses the same password for both the keystore and the signing key inside it).
Tests on a real device - By turning this option on your app is launched on a real device inside Mobile Center’s Test Cloud service, and the test verifies that your app launched successfully. This is a great sanity check that your build has worked and your app starts up. You have no control over which device it runs on—it’s compatible with at least your app’s minimum SDK installed, but other than that you have no control. It also makes your build take longer as you’ll have to wait for an available device. It runs using UITest, and you’ll need to use a release build to prevent the shared Mono runtime from being used. You need a valid Test Cloud subscription for this to work, but as part of your Mobile Center free trial you’ll get a valid subscription; turn this option on.
Distribute builds - These options allow you to distribute your final builds to either beta testers or the store.

Once your build’s configured, click ‘Save.’ This will queue up a first build under the master branch.

_Figure 5. All builds for a branch can be seen by clicking on the branch_

Under each branch is a list of all the builds for that branch, with only one entry—the build that was kicked off by setting up the branch. If you click on the build you can see more information about it, including a full build log.

_Figure 6. Your Android app should build successfully_

Once your app is built you’ll see the build marked as a success, as well as how long the build took and logs detailing everything that happened. The build has taken the code from your source control provider, restored the NuGet packages you use, compiled your Android app, signed the APK using your keystore, installed it on a device, and launched it to verify that everything worked. You’ll also have an option to download the Android APK and build logs if you want. If your app didn’t build, check the build output and fix whatever’s causing the issue.

This is our Android app setup and built, now let’s setup the iOS app.

Configuring the iOS build

Although we’re building a cross-platform Xamarin app, we’re building two apps, one iOS and one Android. This means we must set up our iOS app as a separate app in Mobile Center. Repeat the same steps as for the Android app, but call the app Countr - iOS and set the OS to iOS.

_Figure 7. iOS builds have a few more options than Android builds_

As before select the master branch to setup the build. The ‘Build app’ section has a few different options to the Android version.

Build app
Project - Unlike Android, from here you can only select the solution to build, not the project. This means to avoid confusion you should only have one iOS app in your solution—if you have more than one, it might be worth breaking them out into separate solutions. This should default to Countr.sln.
Configuration - This is the same as the Android configuration section. To be able to run a launch test on our builds using UITest we need to use the Debug configuration; set this to Debug instead of Release.
Xcode version - You can configure which version of Xcode to use when building. Only recent versions are available (at the time of writing, 8.3.2 is the latest and you can only select versions starting at 8.0), but this is useful if you have a reason for needing an earlier version. We can leave this as the default which is the latest released version.
Build type - Unlike Android, iOS builds come in two types: simulator and device builds. Set this to Device build to run a launch test on a real device (launch tests aren’t available on simulators).
Build frequency - Leave this as Build this branch on every push.
Sign builds - Mobile Center can sign our iOS builds using our certificate and provisioning profile.
Provisioning profile - The easiest way to set the provisioning profile is to head to the Apple developer site, log in, head to the provisioning profiles section of ‘Certificates, identifiers and profiles,’ and download your developer profile. Once downloaded, upload it to Mobile Center.
Certificate - Mobile Center needs your signing certificate as a .p12 file. You can’t download this from the Apple developer website, instead you need to export it from your keychain. Open ‘Keychain Access’ on your Mac and search for a certificate with a name that starts with ‘iPhone Developer.’ Select this certificate and click ‘File→Export Items…’, enter a name, select a sensible location, and click Save. When prompted, enter a password to use to encrypt this certificate.

_Figure 8. Use Keychain Access to export your certificate_

From Mobile Center upload this .p12 file and enter the password you used.

Tests on a real device - Turn this option on again to sanity check out app on a real iOS device.
Distribute builds - Leave this off for now. Once your build is set up, click Save and watch the logs to see it build and run the launch test. Again, if the build fails check the build output and fix any issues.

That’s the basics using Mobile Center. If you want to learn more about making cross-platform mobile apps using Xamarin, download the free first chapter of Xamarin in Action and see this Slideshare presentation.

Creating Visual Studio project and solution templates - Part 1, dotnet new

Wed, 25 Oct 2017 06:42:53 +0000

I’ve recently updated my MvvmCross templates to support .NET Standard to be ready for the awesome future. Unfortunately this meant a complete rewrite of my templates as the out of the box Visual Studio project template extensions on both Windows and Mac don’t support .NET Standard. As it turned out though, this rewrite was a blessing in disguise as there is a new way to do templates, and this can be used with the dotnet cli as well as inside extensions for VS on Windows and Mac, meaning I can now create one template and use it on all platforms.

This post shows how to create templates for the .NET cli, and in the next two posts these templates will be ported to Visual Studio, first Windows then Mac.

dotnet new

As part of the new .NET Core awesomeness, there is a new cli for .NET. You can install this manually from https://www.microsoft.com/net/core, otherwise you can install it with Visual Studio.

Once you have the .NET cli installed you can use the command line to create new apps based off templates using dotnet new <template name>. Some templates come pre-installed, and you can download and install others to provide any kind of project your heart desires. This post is not going to cover the cli, you can read up on that from the previous link, instead we are going to look at creating templates.

dotnet new templates

The templates used by dotnet new are actually really simple - they are fully working projects or solutions with a config file. If you’ve ever created a project or solution extension for Visual Studio you’ve probably had the fun of creating a project then changing namespaces and project references to be $ProjectName or other such replacement tokens. These projects then cannot be run, so to change them you have to change the code, build the extension, create a project, find problems, rinse and repeat. With dotnet new templates you don’t have to do this.

Step 1 - create a project

The first thing to do is a create a project or solution that you want to create the template from. Add the relevant NuGet packages, add classes, resources, whatever you need. This is a normal project, the same you use every day so you can run it at any time for testing.

Step 2 - create a template.config file

At the root of your project or solution, create a folder called .template.config and add a json file called template.json to that folder. This file defines the structure of the template, including providing a base namespace that you used in your project. When a new project is created using your template, all the files in your source project will be used, with the namespace changed to the namespace passed to dotnet new.

For example if your project had the namespace MvvmCrossApp, you set this in the template.json file. If the end user tells dotnet new to use the namespace Foo, any occurrences of MvvmCrossApp in source files or project files will be replaced with Foo automatically - no need to use replacement tokens.

You can read about all the fields in this file at https://blogs.msdn.microsoft.com/dotnet/2017/04/02/how-to-create-your-own-templates-for-dotnet-new/. You can also install an extension to Visual Studio for Windows called Sidewaffle Creator that has a wizard to help create these files.

Step 3 - create a NuGet package

These templates are packaged as NuGet packages, so you will need to create a .nuspec and package up your files. This NuSpec needs to have the usual id, description, version, as well a package type of template set in the metadata. All the source files for your project need to be added as files. You can see an example I use for my MvvmCross templates below:

<?xml version="1.0" encoding="utf-8"?>
<package xmlns="http://schemas.microsoft.com/packaging/2012/06/nuspec.xsd">
<metadata>
<id>MvvmCross.Templates.CSharp.Native.SinglePage.iOS.Android</id>
<version>1.0.0</version>
<description>
Creates an example single page MvvmCross app for iOS, Android, UWP and WPF.
</description>
<authors>Jim Bennett</authors>
<packageTypes>
<packageType name="Template" />
</packageTypes>
</metadata>
<files>
<file src=".\MvvmCrossNativeSinglePage\**" target="content" exclude=".\MvvmCrossNativeSinglePage\packages\**;.\MvvmCrossNativeSinglePage\*.UWP\**;.\MvvmCrossNativeSinglePage\*.WPF\**;.\MvvmCrossNativeSinglePage\.template.config\template.json;**\bin\**;**\obj\**;**\.vs\**;**\*.user;"/>
<file src=".\MVVMCross.XSAddIn\MvvmCrossNativeSinglePage.template.json" target="content\.template.config\template.json"/>
</files>
</package>

This is packaged up using the NuGet command line to give a standard NuGet package.

Step 4 - install the template

Templates can be installed in one of two ways. To test locally, you can install using:

dotnet new --install <path_to_nuget_package>

Then if you run dotnet new you should see your template in the list of available templates. You can then create a new project or solution using your template by using:

dotnet new <template_short_name> -o <output_folder_and_namespace_name>

Step 5 - distribute the template

Once you’ve verified locally, you can upload your NuGet package to NuGet.org. From there anyone can install your template using:

dotnet new --install <NuGet_package_id>

You can see an example of this for my MvvmCross templates at https://github.com/jimbobbennett/MvvmCross-Templates

In the next post we’ll look at using this template inside a Visual Studio for Windows extension.

Hitting refresh on my career

Tue, 24 Oct 2017 06:56:55 +0000

Inspired by Satya Nadella’s Hit Refresh, a book well worth reading, I thought I’d write about my Hit Refresh moment.

Hit Refresh is inspired by refreshing your browser - it doesn’t reload everything, instead it just reloads part of the page. This is reflected in how we can change our lives, not by a complete change, but by refreshing a part. In my case it was three and a half years ago. I was working in finance for some of the worlds largest institutions writing desktop apps, and I was bored. By doing boring work using old technology for people who only cared about how much money they could make I’d lost my passion. No longer did I code at weekends or keep up to date with the latest cool things. In the end, enough was enough. I needed a change.

I decided mobile was the way to go, as I felt a strong interest in these new paradigms - the ability to take our content creation and consumption with us wherever we went. Looking around the technology landscape I discovered Xamarin. I could write mobile apps using C#, a language I was very familiar with, in Visual Studio, a tool I was also very familiar with. In a short space of time I took the plunge - I quit a job I hated and parked in a co-working space in Thailand with my MacBook, a Surface and a Xamarin license (and a very understanding wife and toddler). I spent 4 months learning this new, exciting technology.

At that point I couldn’t find a job as a Xamarin developer, instead I went back to finance to pay the bills, but got heavily involved in the community. My passion was back so I wanted to spread this love around the world. I published code again, I blogged again, and for the first time I spoke at events, starting with meetups, then moving to international conferences. I’d truly found my passion. Later I got a job as a Xamarin developer, moving to New Zealand. Here I got involved running the local meetups, spending time with the local Microsoft team helping at all day events. I even ended up writing a book on Xamarin!

I went from bored, to full of passion for a technology that made mobile so easy. I’d learned how much I loved presenting, so do it at every opportunity. I’d reinvented myself, not by a conscious effort to become someone else, but by discovering who I truly am. I am more content with my life, I love what I do. And career-wise, it has played out incredibly well, with another Hit Refresh moment coming, that I will blog about soon!

There is a moral to this story - find what you love and use it to hit refresh on your life, who knows where it will lead!

This post is part of the collection of #HitRefresh moments. Read more at www.hitrefreshbook.com

Using LiveXAML to improve your Xamarin Forms UIs

Tue, 17 Oct 2017 07:53:52 +0000

One of the downsides to UI development is how long it takes to get a good looking UI. You spend an inordinate amount of time setting up the UI, running your app to see how it looks, tweaking the UI, running again, tweaking again, running again. Then of course you have to repeat on multiple devices with different OSes or screen sizes.

For Xamarin Forms there are a number of tools to help you when building UIs using XAML. The one that comes out of the box is the previewer - a preview of what your XAML will look like on iOS or Android, phones or tablets. This preview updates as you update your XAML but is limited. If your UI relies on bound data you can’t see the full experience, and anything that relies on resources doesn’t always work so well.

Another good tool is the Xamarin Live Player, which I’ll look at in a later blog post.

Currently though the tool that is interesting me the most is LiveXAML for Xamarin Forms from Mihhail Maslakov - http://www.livexaml.com.

This tool is an extension for Visual Studio (both Windows and Mac which is cool) that injects some code inside your app to listen for XAML updates. You update a XAML file inside Visual Studio and the extension tells your app to update the UI. This means you can tweak your code and instantly see the updates in your actual app running under real world conditions. Essentially magic!

Lets now see it in action. Start by installing the LiveXAMl extension from the extensions gallery, then create a new blank Xamarin Forms app, adding the LiveXAML NuGet package to your core Forms project.

The out of the box forms template creates a single page app with a label. saying “Welcome to Xamarin Forms!”. If you run your app you will see this label. Try changing the text color in your XAML page and saving it - you will see the text color update on your app.

Magic! This is a very simplistic example, but the power here is obvious. I’ve wasted countless hours making minor tweaks, especially to margins, font sizes and colors, time that could be saved by using LiveXAML.

Lets look at a more complicated example, using the sample app I built for a talk at NDC Sydney and MonkeyFest on Xamarin Forms effects and custom renderers. If I change a property on an effect, the UI updates.

This works for pages and styles - so if you update a style in your app.xaml file, the UI will update, something very important if you use styles a lot (which I do). It also works for controls, so if you have a control defined using XAML on your page, updating the control updates the page.

You can also use Visual Studio for Mac’s run configurations to run two apps at the same time, such as an iOS and an Android app, then update a XAML page and see the UI update on both platforms. Another huge time saver.

You don’t just have to use emulators, you can use physical devices. This happens via WiFi over port 53030 and 53031, so if you are on a corporate WiFi network you’ll need to have these ports open. I couldn’t get this working at the office, but it worked at home.

Support seems to be pretty good as well. When I first tried it out I had issues with resources and controls not working, but the latest NuGet package seems to have fixed everything, having regular releases adding new features is always a good sign.

As great as this tool is, there is a downside. As the name suggests it is XAML focused, so updates to C# code will not be rendered on screen.

Overall opinion - this is great. It’s a paid for tool, but has a free version to allow you to try it out. I would say it’s probably worth the money, especially if your company pays for it as the time you save would more than pay for the license cost.

Introduction to UI Testing with Xamarin

Sun, 20 Aug 2017 04:10:12 +0000

This article is an excerpt from Xamarin in Action. Save 37% off the cover price using code fccbennett at http://xam.jbb.io.

One of the great things about the MVVM design pattern is that it allows us to maximize the code in our cross-platform model and view-model layers. This means we’ve written the bulk of our code and we’ve also written unit tests for it, giving us a degree of confidence that our code works. This article introduces UI testing in Xamarin.

Unit tests are great, but they don’t cover two important areas - have we used the correct controls on our view and bound them correctly, and does our app run on the device. It’s great to have a property on a view model that we bind to a text field to allow the user to enter the name of a counter, but what if we accidentally use the wrong control, such as a label instead of a text box, or even use the right control but forget to add the binding code? What if we’ve used a feature that was only added to the Android SDK in API 21 but our app manifest shows our app runs on API 19 and higher? This is where UI testing comes in - it allows us to run our app on emulators, simulators, and devices, and to write automated tests in code against it.

The concept behind UI testing is simple - run your app and have something interact with it using the user interface components (such as tapping buttons or entering text in text boxes), and validate that everything is working by ensuring the app doesn’t crash and that the results of the users’ actions are shown on the UI as expected. This kind of testing started out life for desktop apps, where the aim was to make testing more reliable and cheaper - after all, humans are expensive and after testing the same screen many, many times they can get bored and make mistakes or miss problems. Automated UI testing also allowed for better time usage with tests being run overnight and developers discovering if they’ve broken anything the next morning.

For desktop apps, UI testing was reasonably simple - launch the app and test it, maybe testing on a few different screen sizes, but always on one OS with maybe one or two different versions as desktop OSes don’t change often. With mobile, things are more complicated. We want to support two major OSes with our cross-platform app, with multiple versions. We also have different hardware with different screen sizes. On iOS this isn’t too bad - we only need to support a small number of OS versions (maybe the current one and the previous) and a small number of different devices, but on Android, as we’ve already seen it’s a mess with multiple OS versions in regular use, a huge range of screen sizes available, and worst of all customizations to the OS from both the hardware manufacturer and carrier.

This is why UI testing is hugely important for mobile apps. A human can’t test on a wide range of devices without needing a lot of time or lots of humans involved in the process (expensive) and without them all going mad as they install the app on another device and run the same test for the millionth time.

Writing UI tests using Xamarin UITest

You write UI tests in the same way that you write a unit test – you decide what you want to test, then write some code to create a test. This code uses some kind of framework that is able to launch your app and interact with it as if it was a real user. Many different frameworks are around for testing, and the table below shows some of these.

Framework	Platforms	Language	Description
Espresso	Android	Java	Google’s testing framework for Android apps has deep integration with Android. https://google.github.io/androidtesting-support-library/docs/espresso/
XCTest	iOS	Objective-C/Swift	Apple’s UI testing framework has deep integration with iOS. https://developer.apple.com/reference/xctest
Appium	iOS/Android	Any (Java/C#/PHP/Ruby etc.)	Open source cross-platform testing framework based off of Selenium (a web UI testing framework). http://appium.io
UITest	iOS/Android	C#/F#	Xamarin’s testing framework heavily integrated into Visual Studio for Windows (Android) and Mac (iOS and Android). https://developer.xamarin.com/guides/testcloud/uitest/

For this article, I’ll focus on Xamarin UITest, which is well integrated into Visual Studio. For testing Android apps, you can use either Windows or Mac, but for testing iOS apps you’ll need to use a Mac - it’s not supported on Windows at the moment.

Xamarin UITest is based off a testing framework called Calabash which was written in Ruby and is fully open source and maintained by Xamarin. UITest is a layer on top of this that allows you to write your tests in C# and run them using NUnit. These tests are written in the same way as unit tests using the arrange, act, assert pattern, with the arrange part launching the app and getting it to the relevant state ready to test, the act interacts with the UI as if it was a user, and the assert queries the UI to ensure it’s in the correct state.

UI tests, like unit tests, follow the arrange, act, assert pattern

Setting up your app for UI testing

For this section, we’ll focus on the Countr app, as there’s more to test here. You can download the source code from this book here, so download this and open the completed Countr solution from chapter 13. When we built the model layer in our app, we added a new unit test project that we used for unit tests for both the model and view model layers. For our UI tests, we also need a new project that contains and runs our UI tests.

Creating the UI test project

Add a new UITest project to the Countr solution using Visual Studio for Windows by right-clicking on the solution, selecting ‘Add→New Project…’ and from the ‘Add new project’ dialog select ‘Visual C#→Cross-Platform’ on the left and select ‘UI Test App’ from the middle. For Mac, right-click on the solution and select ‘Add→Add New Project…’, select ‘Multiplatform→Tests’ on the left and ‘UI Test App’ from the middle and tap ‘Next’. Name your project ‘Countr.UITests’ and click ‘OK’ (Windows) or ‘Create’ (Mac).

Adding a new UITest project using Visual Studio for Mac

Adding a new UITest project using Visual Studio 2017

Once the test project has been added it’ll install two NuGet packages that UITest needs - NUnit and Xamarin.UITest. It’s worth updating the Xamarin.UITest NuGet package to the latest version, as they often push out bug fixes to ensure it works on the latest mobile OS versions. Don’t update NUnit. UITest only works with NUnit 2, not NUnit 3 and if you update this package your tests won’t work and you’ll need to remove the package and re-install NUnit 2.

The UI test project has two files auto-generated for you - AppInitializer.cs and Tests.cs.

AppInitializer.cs: This is a static helper class with a single static method which is used to start your app. UITest has an IApp interface used to represent your running app, and this has methods on it for interacting with the UI elements in your app or to a limited extent the device hardware (for example rotating the device). The StartApp method returns an instance of IApp that your tests can use. This method uses a helper class called ConfigureApp to start the app, and this helper class has a fluent API that allows you to configure and run your app. The auto-generated code doesn’t do much to configure the app, it specifies its type (Android or iOS) based on the platform passed in to the method.
Tests.cs: This file contains an UI test to run. This test fixture has a parameterized constructor that takes the platform to run the tests on as one of the values from the Platform enum, either Platform.iOS or Platform.Android, and has two TestFixture attributes, one for each platform. This means that we have two test fixtures - one Android and one iOS. This fixture has a setup method that uses the AppInitializer to start the app before each test, and a single test that calls the Screenshot method on the IApp returned from the app initializer to take a screenshot.

Setting up your Android apps for UI testing

By default, Xamarin Android apps are configured in debug builds to use the shared mono runtime. When you deploy your app to a device or emulator, time is taken copying the code over, and anything that can make your app smaller is a good thing as this reduces the time to install. Xamarin Android apps use a mono runtime (mono being the cross-platform version of .Net that Xamarin is based on) to provide the .Net framework, and this is a large piece of code bundled in your app. Rather than bundling it in, for debug builds you can use a shared version which is installed separately, making your app smaller. Unfortunately, when doing UI tests you can’t use the shared runtime, and you’ve two options:

Don’t use the shared runtime: You can turn off the shared mono runtime from the project properties. In Visual Studio for Windows you’ll find it in the ‘Android Options’ tab at the top of the ‘Packaging’ page, on Mac it’s on the ‘Android Build’ tab at the top of the ‘General’ page. Untick the ‘Use Shared Mono Runtime’ box to turn this off, but be aware that this increases your build times.
Release builds: Release builds don’t have the shared mono runtime turned on. After all, when you build a release version it’s usually for deployment such as to the store, and your users won’t have the shared mono runtime installed. The downside to using a release build is that you need to grant your app permission to access the internet. This isn’t a problem if your app already accesses the internet, but if it doesn’t you many not want to ask your users for this extra permission as they might not want to grant it. If you want to use a release build, then you can grant this permission in Visual Studio by opening the project properties, heading to the ‘Android Manifest’ tab and finding the INTERNET permission in the ‘Required Permissions’ list and ticking it. On Mac, double-click on the AndroidManifest.xml file in the Properties folder and tick the permission.

Adding the internet permission to the AndroidManifest.xml

Setting up your iOS apps for UI testing

Apart from the shared mono runtime, out of the box UITest works with Android – UITest is able to connect to your running Android app on a device or an emulator and interact with it. iOS, on the other hand, isn’t quite as simple. Due to the stricter security on iOS you can’t connect to a simulator or device and interact with the app. Instead we need to install an extra component into your iOS apps that you initialize before your UI tests can run. To do this, add the Xamarin.TestCloud.Agent NuGet package to the Countr iOS app (Test Cloud is the Xamarin cloud-based testing service – you’ll see the name used in a few places with UITest).

Adding the Xamarin Test Cloud Agent NuGet package

Once this NuGet is installed you’ll need to add a single line of code to initialize it. Open AppDelegate.cs and add the following code:

public override bool FinishedLaunching(UIApplication app,
NSDictionary options)
{
#if DEBUG
Xamarin.Calabash.Start();
#endif
...
}

This code starts the Calabash server for debug builds only. The Calabash server is an HTTP server that runs inside your app, and the UITest framework connects to this to allow it to interact with your app. Apple is strict about security in their apps, and they’d never allow an app with an open HTTP port like this on the app store. To avoid this the Calabash server is only enabled for debug builds - for release builds this code won’t get run, the linker strips it out and your app won’t get rejected from the app store (at least not due to the Calabash server).

Running the auto-generated tests

UI tests are run in the same way as any other unit test - you can run them from the test pad/explorer. UI tests rely on having a compiled and packaged app to run, and the first step is to build and either deploy to device, emulator or simulator or run the app you want to test. Note that doing a build isn’t enough for Android – a build compiles the code, it doesn’t package it up. The easiest way to ensure you have a compiled and packaged app is to run it once. On Android build in release, for iOS you need to use a debug build to enable the Calabash server. You also need to set what device, emulator or simulator you want to run your tests on in the same way that you’d select the target for debugging.

Getting ready to run the tests

If you open the test pad or explorer you may not see the UI tests if the project hasn’t been built. If you don’t see the tests then build the UITest project and you should see the tests appear. If you expand the test tree in Visual Studio for Mac you’ll see two fixtures - Tests(Android) and Tests(iOS).

The Test Explorer (Windows) and Test pad (Mac)

Two fixtures are declared with the two TestFixture attributes on the Tests class. When you run the tests from Tests(Android) it constructs the test fixture by passing Platform.Android to the constructor, which in turn uses the AppInitializer to start the Android app. Tests(iOS) is the same, but for the iOS app. Under each fixture you’ll see the same test called AppLaunches.

Grouping tests in the Visual Studio test explorer

In Visual Studio, you don’t see the same hierarchy out of the box; drop down the ‘Group By’ box and select ‘Class’ to see the tests grouped by test fixture. You can only test Android apps using Visual Studio. Feel free to comment out the [TestFixture(Platform.iOS)] attribute from the Tests class to remove these tests from the explorer.

Before we can run the test, we need to make a small tweak. Despite the test calling app.Screenshot, this test won’t spit out a screenshot. For some reason, out of the box UITest is configured to only create screenshots if the tests are run on Xamarin’s Test Cloud, and we need to change the configuration to always generate screenshots. To do this, add the code in the following listing to AppInitializer.cs.

public static IApp StartApp(Platform platform)
{
if (platform == Platform.Android)
{
return ConfigureApp
.Android
.EnableLocalScreenshots()
.StartApp();
}
return ConfigureApp
.iOS
.EnableLocalScreenshots()
.StartApp();
}

By default, the StartApp method doesn’t do anything to configure the app which is being tested, and this means that it expects the app to be tested to be a part of the current solution. You also need to configure UI test to know which apps in the solution to use as there could be multiple.

Setting the app to test in Visual Studio for Mac

Open the test pad and expand the Countr.UITests node. Under this you’ll see the test fixtures, as well as a node called Test Apps shown next to a stack of green arrows. Right-click on this and select ‘Add App Project’. From the dialog that appears tick ‘Countr.Droid’ and ‘Countr.iOS’ and click ‘OK’. You’ll see these two apps appear under the ‘Test Apps’ node. If you right-click on one of them you’ll see a number of options, including a list of possible target devices to run against, with ‘Current device’ ticked. This list is used to set which device the UI tests are run against when you run them from the pad. If you leave ‘Current Device’ selected it’ll use whatever target is set from the main toolbar, but if you always want the tests to run against a particular emulator, simulator or device you can select it from here.

Setting the app to test in Visual Studio 2017

UITest uses NUnit to run tests, and we need to ensure Visual is configured to run NUnit tests. As I mentioned, to use UITest we need to install the NUnit 2 adapter. From ‘Tools→Extensions and Updates’, select the ‘Online’ tab on the left and search for ‘NUnit 2’. Click on ‘NUnit 2 Test Adapter’ in the list in the middle and click the ‘Download’ button. You’ll need to close Visual Studio for this to be installed; relaunch it after the install and re-load the Countr solution.

Installing the NUnit 2 test adapter

Visual Studio only supports testing Android; delete the [TestFixture(Platform.iOS)] attribute from the Tests class. This stops iOS tests showing up in the test explorer.

Unlike Visual Studio for Mac, there is no way on Windows to set the test apps. Instead we need to configure this in code by giving it the path to the Android ‘APK’, which is in the output folder and is named based on the Android package name with the file extension .apk, with release builds having the suffix of ‐Signed to indicate that they’ve been signed with our keystore. We set the package name in the Android manifest in the last chapter based off a reverse domain name (mine was set to io.jimbobbennett.Countr), and you can find this file in Countr.Droid\bin\Release (assuming you’ve built using the release configuration) or Countr.Droid\bin\Debug for the debug configuration. We’ll be using release Android builds for the purposes of this book; add the code in the following listing to point UITest to the right APK, substituting in your package name.

if (platform == Platform.Android)
{
return ConfigureApp
.Android
.EnableLocalScreenshots()
.ApkFile ("../../../Countr.Droid/bin/Release/<your package name>‐Signed.apk")
.StartApp();
}

When tests are run they’re run on the device or emulator that you selected for the build configuration, the same as for debugging your apps. This makes it easy to change which device tests are run on by changing the dropdown in the toolbar, the same way you’d change the target for debugging.

Running the test

Once the test apps are configured you can run the tests by double-clicking on them in the test pad in Visual Studio for Mac, or by right-clicking on them in the Visual Studio for Windows Test Explorer and selecting ‘Run Selected Tests.’ If you’re testing Android set the build configuration to release, for iOS set it to debug.

The device to test on is set in the same way as the device for debugging

That’s all for UI testing. If you want to learn more about making cross-platform mobile apps using Xamarin, download the free first chapter of Xamarin in Action and see this Slideshare presentation.

Firewall issues with iOS simulators

Thu, 10 Aug 2017 01:19:54 +0000

I keep getting an annoying issue with my iOS simulators. When I run an app in a simulator that needs any form of network connection I get a dialog box popup from the built in Mac firewall asking if I want to allow incoming connections. Even though I click ‘allow’, I get asked this every time and it’s getting tedious.

The issue seems to be in the firewall. When I allow an iOS app it permits the app to access the network, but the iOS simulator does not seem to have permissions. The firewall gets confused - the simulator doesn’t have permissions so the network request is blocked pending approval from me, but the firewall doesn’t ‘see’ the simulator, instead it sees the iOS app and gives that permission.

The problems comes from Xcode, it should set the firewall permissions when you install it but sometimes it fails. There is an easy fix - I’ve got a script that opens up the firewall to the simulator. It turns the firewall off (so make sure you are disconnected from the internet or not anywhere dodgy), enables Xcode and the iOS simulator as a firewall exception, then turns it back on.

# temporarily shut firewall off:
sudo /usr/libexec/ApplicationFirewall/socketfilterfw --setglobalstate off

# put Xcode as an exception:
/usr/libexec/ApplicationFirewall/socketfilterfw --add /Applications/Xcode.app/Contents/MacOS/Xcode

# put iOS Simulator as an exception:
/usr/libexec/ApplicationFirewall/socketfilterfw --add /Applications/Xcode.app/Contents/Developer/Applications/Simulator.app/Contents/MacOS/Simulator

# re-enable firewall:
sudo /usr/libexec/ApplicationFirewall/socketfilterfw --setglobalstate on

Run this, give it your password as it needs sudo access and bingo, problem goes away. You may need to run this every time Xcode is updated though…

Game of Phones - Winter is coming

Wed, 07 Jun 2017 07:38:09 +0000

Winter of Xamarin - register now at http://aka.ms/winterofxamarin.

Stay warm with Microsoft & Xamarin!

Thu, 01 Jun 2017 08:16:52 +0000

For those in New Zealand, Microsoft and Xamarin have announced some cool events:

Brr! It’s getting cold outside. Microsoft has a plan to get you through winter with some awesome Xamarin events and competitions and apparel!

Are you ready to sharpen your skills in a fun, fast-paced environment? Microsoft is hosting FREE Xamarin training events around New Zealand! Xamarin lets you write code once for a mobile app and deploy it to Android, iOS and Windows in one go. We’ll teach you how to build on the same tool top Fortune 500 companies use, and help you make your next big app!

To keep you warm, we’ve got limited edition Xamarin Beanies for every event attendee, completed with limited edition Xamarin socks for every competition entry. The top app developers also get Xbox Ones and hard cash to splurge over the winter months!

Agenda for the Events

9:00am – 9:30am Registration 9:30am – 10:00am Introduction to Xamarin 10:00am – 11:00am Building Cross Platform Apps 12:00pm – 1:00pm Lunch 1:00pm – 4:30pm Xamarin App Hackathon

Dates and Locations

Auckland Event 1 - http://aka.ms/woxakl1 Friday 16th of June - Beca House Auditorium, 21 Pitt Street,

Auckland Event 2 - http://aka.ms/woxakl2 Saturday 17th of June - Beca House Auditorium, 21 Pitt Street,

Tauranga Event - http://aka.ms/woxtaudev Friday 9th of June Toi-Ohomai, 200 Cameron Road, Tauranga

Christchurch Event - http://aka.ms/woxchch1 Saturday 24th of June Ara Polytechnic, Madras Street Campus,

Specflow support in Visual Studio for Mac

Thu, 01 Jun 2017 08:10:50 +0000

Love Specflow and Visual Studio for Mac? Well you are in luck as there is now a working extension for Visual Studio for Mac to enable full specflow support including generating feature files with syntax highlighting.

It’s all open source and built upon the amazing work by Mike Jones. You can find it on GitHub at https://github.com/straighteight/SpecFlow-VS-Mac-Integration or in the VS extension gallery:

As always a big thank you to Mikayla Hutchinson for her continued help in getting this all working.

Opening multiple solutions in Visual Studio for Mac

Tue, 23 May 2017 23:28:34 +0000

One area Macs are very different to Windows PCs is in the way documents are opened. Mac apps manage documents internally instead of being able to tap a start button again and open a second instance of your app to load a new document.

This means that on Windows Xamarin developers can open multiple instances of Visual Studio at the same time, allowing them to have multiple solutions open. On the Mac this ability has been missing - you click VS again and it just focuses on the same instance that’s already running. There have been a number of workarounds for Xamarin Studio and Visual Studio the apps you can run that will launch another instance, but actually - you don’t need to!

Unlike VS on Windows, VS on Mac (and Xamarin Studio for those who haven’t updated) can open multiple solutions at the same time in the same window. The option to do so is a bit hidden, but it’s there.

What you do is:

Open your first solution
Select ‘File->Open’
Select your solution in the Open dialog (just a single click, don’t double click to open it)
Click the ‘Options’ button
Uncheck ‘Close current workspace’

Now click ‘Open’

Viola! Multiple solutions in the same solution pad.

If the solution is in your recent list you can also open it in the same workspace by holding down the Control key whilst clicking on the solution. Thanks to Lluis Sanchez Gual for this one.

@CodeMillMatt A little trick: to quickly open a solution without closing the current one, hold Control while clicking on a file in Recent Solutions
— Lluis Sanchez Gual (@slluis) May 25, 2017

Upcoming talks

Wed, 10 May 2017 08:48:31 +0000

I’m excited to announce that I’ll be talking at a couple of major conferences this year:

MonkeyFest 2017

NDC Sydney

At both of these conferences I’ll be talking about adding platform specifics to Xamarin Forms apps. Hope to see you there!

Nugetizer 3000

Wed, 10 May 2017 07:58:00 +0000

One cool new feature in Visual Studio for Mac is the ability to create NuGet packages from your projects automatically whenever you compile. This new feature is thanks to a cool project started by Xamarin called the Nugetizer 3000 (I do love Miguel’s naming conventions).

It’s really easy to turn this on. From your project open the project properties and head to the ‘NuGet Package -> Build’ tab. From there tick the ‘Create a NuGet package’ box.

Once this is ticked you need to set up the metadata for your package, and this can be done from the metadata tab. You have to fill in everything in the ‘General’ tab, and you can optionally fill in the ‘Details’ tab.

Once you click ‘OK’, Visual Studio will add a new NuGet package to your project - NuGet.Build.Packaging. This package doesn’t contain any assemblies, it just contains build targets to create the NuGet package.

Once this package is installed you can compile your project and it will spit out a nupkg file in the projects output directory. This package has everything all set up for it, including dependencies on all the packages and framework libraries used by your project.

This makes package creation easy as there is no faffing around with nuspec files and command lines. You can even control values such as the version by configuring project properties when calling MSBuild from the command line, very useful if you want to tweak the version number at build time.

Be quick - 50% off Xamarin In Action for one week

Fri, 24 Feb 2017 17:55:54 +0000

The MEAP of my book, Xamarin In Action, has just been updated to include a new chapter covering building cross-platform view models. This chapter talks about how you can use view models to increase the amount of cross-platform, unit testable code in your Xamarin app by moving UI logic away from the platform specific layers down into your cross-platform code.

To celebrate you can get 50% off my book for one week only. Head to http://xam.jbb.io and use the code mlbennett.

Not sure if this is the book for you? Well if you are building or plan to build Xamarin apps then yes, this is the book for you. Rather than being a dry reference book that replicates what is in the on-line docs, this instead teaches you how to build and ship production quality Xamarin apps from idea to the store. I teaches MVVM as a way to ramp up the amount of cross-platform code in your app - after all large amounts of code sharing is the killer feature of Xamarin!

This book is still being written, so if you buy it now you get the first 8 chapters, with new chapters as they are being written, and you can get involved with making the book better by giving feedback and suggesting improvements.

You can download the first chapter for free from here. You can also read an excerpt all about MVVM on DZONE, Medium or my own blog (whichever is your preference).

The current table of contents is:

Part 1 - Getting started with native cross-platform apps

1 Introducing native cross-platform applications with Xamarin 2 Hello MVVM 3 MVVM – a deeper dive 4 Hello MVVM – a deeper dive 5 What are we (a)waiting for? - An introduction to multithreaded code

Part 2 - From blank solution to an empty, working app

6 Designing MVVM cross-platform apps 7 Building cross-platform models 8 Building cross-platform view models 9 Building platform specific views - Android 10 Building platform specific views - iOS 11 Running the app

Part 3 - From a working app to a production ready app

12 Testing the app using UITest 13 Instrumentation and monitoring 14 Deploying the app

Appendices

A1 What are design patterns anyway? A2 Using someone else’s code A3 UI flows and threads for SquareRt and Countr

MvvmCross extension for Visual Studio 2017

Fri, 20 Jan 2017 21:50:37 +0000

Update - it’s now on the marketplace!

Link here: https://marketplace.visualstudio.com/items?itemName=JimBobBennett.MvvmCrossforVisualStudio

The awesome team at Microsoft have got it all working, so I’ve managed to upload the VS2017 version to the marketplace. You can now download it from Extensions and Updates.

Thanks to everyone on the VS extensions team who helped make it all work!

Visual Studio 2017 is now available as a release candidate, so I thought it was time to update my MvvmCross extension to support it.

The update was really, really easy - and is well documented by the VS team. Unfortunately there is a bug in the marketplace, so although the upgraded extension works, I can’t upload it. Microsoft are aware of this issue and are working to fix it, so once they done so I’ll make the update available through the Extensions and Updates menu item.

For now though you can download the VSIX from my GitHub page and manually install it.

https://github.com/jimbobbennett/MvvmCross-VisualStudioAddIn/releases/tag/v1.2.0

MVVM – the design pattern for Xamarin apps

Thu, 12 Jan 2017 23:09:30 +0000

This article is an excerpt from Xamarin in Action

MVVM is the most popular design pattern for cross-platform apps built using Xamarin, and has a history of being a very successful design pattern for building Windows desktop apps using WPF, WinRT apps, and Windows 10 UWP apps – even web frameworks like knockout.js are using it. When Xamarin designed Xamarin.Forms (where the goal was to have as much code sharing as possible), the principles of MVVM were baked into the underlying framework right off the bat.

For the purposes of this article, let’s consider a simple square root calculator app called “Sqrt” that has a text box you can put a number into, and a button. When you tap the button it calculates the square root of the number in the text box and shows this on a label. An example of this app is shown below.

A simple square root calculator app that calculates the square root of a given number

The simplest way to write this app is to wire up the button to an event that takes the value directly from the text box, calculates the square root, and writes the value to a label. All this can be done in the code behind file for the UI. Simple, and all in one class. Below is some pseudo-code for the kind of thing you might write.

MyAddButton.Click += CalcSquareRoot; ❶
...
private void CalcSquareRoot(object sender, EventArgs args)
{
var number = double.Parse(NumberTextBox.Text); ❷
var sqrt = Math.Sqrt(number);
MyResultLabel.Text = sqrt.ToString(); ❸
}

❶ We are listening for the Click event of the button

❷ The number comes from reading the value from the Text property of the text box

❸ Once we have the square root the Text property of the label is set directly

Whilst this seems simple, it has a number of flaws.

First, this is not easily testable. Yes, you can launch the app, type in a number, and tap the button, but this takes a long time. We can only test this app by updating the value in the text box and tapping the button – it would be nice if the calculation code was self-contained so we could test it in isolation. It would be better if we could write unit tests so we can programmatically test our code, covering multiple cases – including edge cases such as missing inputs, or large or negative numbers. This way we can run a set of automated tests quickly, which can be repeated every time we change our code.

Second, this is not cross-platform. One of the reasons for building our app using Xamarin is so that we can have parts of our app written in shared code that works on both iOS and Android. If our calculation is wired directly to the view, we can’t do this.

Xamarin apps are written in C#, so you can share any common business logic whilst having a platform-specific UI

As you can see, a Xamarin app has three layers:

The application layer is the small part of the code that makes your code runnable on each platform; it is separate and has different platform-specific implementations for iOS and Android.
The UI layer is also separate, and has different platform-specific implementations for iOS and Android.
The business logic layer is shared between the two platforms.

In the UI layer there are really two layers – the actual UI widgets and some logic around these widgets. For example, we could put some logic around our answer label to make it only be visible once we have calculated a square root. This expands our three layers to four. Let’s explore ways to maximize code re-use between layers.

To maximize code re-use, it would be good to have UI logic in shared code

To increase the amount of code sharing, it would be great to be able to move the UI logic into shared code as well. The image above shows how the layers would look if we could do this. If we did this, the label in our example would be in the UI layer, and the logic to decide whether it should be visible or hidden would be in the cross-platform UI logic layer. This is a great way to do things – we’re maximizing our code re-use by abstracting our UI logic into cross-platform code.

MVVM has a model, a view model, a view, and a binding layer that keeps the view and view model in sync and connects events on the view to the view model

MVVM helps split the UI from its logic. This pattern’s name is based off the three layers that you use in your app. Let’s look at these layers in the context of our calculator example:

Model - your data and business logic The model contains the number, the logic to calculate the square root, and the result.
View - the actual UI, buttons, text controls and all other widgets This holds the UI widgets – the text box, button and label. It is a passive view so it doesn’t have any code to get or set the values, or handle the events such as the button click.
ViewModel - the UI data and logic

For our calculator app this would have properties that represent the numbers on the model – the input values and the result. It would also have a Command property that wraps the square root calculation logic on the model into an object. The view model would know about the model but have no knowledge of the view.

In addition to these three layers it has a binder, a binding layer that you can think of as glue that connects the view model to the view. This removes the need to write boilerplate code to synchronize the UI – the binder can watch for changes in the view model and update the view to match, or update the view model to match changes made by the user in the UI. This binder is loosely-coupled rather than tightly-coupled, and the connection is often done based on wiring up properties in the view and view model based on their names (so in the case of a binding between a property called “Text” and a property called “Name”, at run time, the binder will use reflection to map these string values to the underlying properties).

Reflecting on reflection

If you’ve never heard of reflection before, it is a part of the C# API that allows you to query details about a class – you can discover properties, fields, methods, the whole shooting match. Once you’ve found out details you can also execute code – for example, you can find a property based on its name then get the value of that property from a particular instance of that class. Reflection is also found in other languages, such as Java – C# reflection is basically the same as Java reflection. This is great for binding – if you bind a property called "Name" the binding code can use reflection to find a property on your view model class with that same name, then it can get the value on your view model instance.

For our calculator app, the binding layer would wire up the text box, button, and label on our UI to the equivalent properties and a command on the view model.

There is a bit of magic to make this binder work, and it is usually implemented in an MVVM framework – a third party library that gives a set of base classes that provide the implementation of this pattern for you.

MVVM frameworks

There are multiple MVVM frameworks around that work with Xamarin native apps such as MvvmCross, MvvmLight, or Caliburn.Micro. Although each one has differences, they all follow the same basic principles and do roughly the same things.

Binding keeps the value in the view in sync with the value in the view model

For example, as shown above, we could have a text box on our calculator app UI that is bound to a Number property. This means at run time it will try to find a public property called “Number” on the view model that it is bound to using reflection, and will show the string contained in that property in the text box. If the user changes the value inside the text box, it will update the value of the Number property to match what the user has typed in. Conversely, if the value of the Number property on the view model changes, the binding will update the text box to match.

The binder doesn’t care about the underlying class type of the view model that you are using, just that it has a public property called “Number” that it can extract the value from. In some of the MVVM frameworks it doesn’t even care if the property is there or not – if it can’t find one, it just treats it as an empty value. This loose coupling is what makes MVVM especially powerful – it allows view models to be completely agnostic to the view, meaning we can write unit tests against the view model that simulate the UI, without worrying about UI code getting in the way. It also supports code re-use, so a view could be glued to any view model that has properties with the names it is expecting.

The different layers of MVVM, and how they fit in with the different layers of a Xamarin app

The image above expands on the previous figures by showing how these layers map to the three layers of MVVM.

The App layer is one that doesn’t really come under the pure MVVM design pattern, but the different MVVM frameworks do provide some application layer features. For this reason, we can have some cross-platform code in our app layer that can control app logic, such as which view is shown first and how the different classes in the app are wired together – for example, code defining which view model is used for each view.
The UI layer is our view layer. This is platform-specific code.
The binding between the UI layer and the UI logic layer is the binder, the glue that connects the UI layer to its logic layer. This is usually a mix of cross-platform and platform-specific code provided by a third party framework.
The UI logic layer is our view model layer, and this provides logic for the UI and other device interactions in a cross-platform fashion. Part of this logic is value conversion – converting from data in your domain objects to data on the UI. For example, you could model a user in your domain with a first name and last name, but want to show the full name on the UI. The view model will provide this value conversion by concatenating the names and giving a single string value that will be shown by the UI.
The business logic layer is the model layer. This contains data, domain objects, logic, and connectivity to external resources such as databases or web services. Again, this is cross-platform.

A quick history lesson

MVVM has been around since 2005 and was developed by two architects from Microsoft, Ken Cooper and Ted Peters. It was primarily created for use with the new UI technology stack coming out of Microsoft, called WPF. It leverages the data binding that was a key feature of WPF. In WPF, you write your UI using XAML, a UI based markup language; in this XAML, you can bind properties of a UI widget to properties defined in the data context of the view – essentially the view model. This allowed UI/UX experts to design the UI using more designer-based tools, and to simply wire up the widgets (based off of names) to code written independently by developers.

If you want to learn more about making cross-platform mobile apps using Xamarin, download the free first chapter of Xamarin in Action and see this Slideshare presentation for a discount code.

MvvmCross Extension for Visual Studio for Mac

Tue, 10 Jan 2017 08:55:01 +0000

I’ve been working to update the MvvmCross Xamarin Studio add-in to work with Visual Studio for Mac. The add-in has been updated thanks to some help from Matthew Robbins, the author of the awesome MFractor add-in, which if you don’t have you MUST get, it’s awesome.

This add-in is now available from the VS for Mac extension manager!

Animating Xamarin Forms progress bars

Thu, 05 Jan 2017 08:43:11 +0000

Don’t you just hate boring UIs? With no animations to bring them to life?

Me too! Especially progress bars - they increment or decrement instantly giving a dull UI.

As you can see from the image above, as the progress bar changes value, the UI updates instantly and it looks a bit dull. What would be nicer is if the progress bar smoothly animated its progress.

Out of the box Xamarin.Forms provides a method to do this - an extension method called ProgressTo. You can call this code from inside your UI to smoothly animate the progress value to whatever value you like. This is great, but it’s not ideal as it relies on the animation happening in the view, so the view needs to know what the value is to animate to. I’m a big fan of MVVM (true - I even written a book on how to use MVVM to build Xamarin apps). The progress value should be in a view model, but then we’d have to wire up a number of UI components to watch view models for changes then start the animation. Ideally it would be better to have a bindable property that we could use to trigger this animation.

We could get this bindable property by creating our own control derived from ProgressBar that has a property for the animation, but that involves a custom control. What would be even better is if we could use the existing progress bar and somehow add a property to it that we can bind to and when the value changes the progress animates to the new value. Happily for us, Xamarin Forms supports this in the form of Attached Properties.

Attached properties are bindable properties, just like the ones you would create in your own controls, except that they can be ‘attached’ to any other control. You define them on one class, and attach them to another. Using these we can create a property on a utility class that is attached to ProgressBar. These properties don’t directly change anything on the class they are attached to, instead you can hook into the property change mechanism to do whatever you need to do.

For example, if we wanted an animated progress bar we could create a new attached property called AnimatedProgress attached to ProgressBar, and every time the value changes we could tell the progress bar to change it’s progress value via the ProgressTo extension method:

public static class AttachedProperties
{
public static BindableProperty AnimatedProgressProperty =
BindableProperty.CreateAttached("AnimatedProgress",
typeof(double),
typeof(ProgressBar),
0.0d,
BindingMode.OneWay,
propertyChanged: (b, o, n) =>
ProgressBarProgressChanged((ProgressBar)b, (double)n));
private static void ProgressBarProgressChanged(ProgressBar progressBar, double progress)
{
ViewExtensions.CancelAnimations(progressBar);
progressBar.ProgressTo((double)progress, 800, Easing.SinOut);
}
}

In this code we are defining a static helper class with the static attached property. This property is called "AnimatedProgress", has a double value, is attached to ProgressBar, has a default value of 0.0. It defaults to bind one way, and this is fine as users cannot interact with a progress bar to change this value. The cool part is the propertyChanged action, this calls a method that animates the progress bar to the new value every time it changes.

Once we’ve defined this property we can attach it to any progress bar we want, using XAML or code behind. The XAML way to set it is:

?xml version="1.0" encoding="utf-8"?>
<ContentPage xmlns="http://xamarin.com/schemas/2014/forms"
xmlns:x="http://schemas.microsoft.com/winfx/2009/xaml"
xmlns:local="clr-namespace:AnimatedProgress"
x:Class="AnimatedProgress.AnimatedProgressPage">
<StackLayout Padding="20,100" Spacing="40">
<ProgressBar local:AttachedProperties.AnimatedProgress="{Binding Progress}"/>
<Entry Text="{Binding ProgressPercent}"/>
</StackLayout>
</ContentPage>

In this XAML we define the local XML namespace pointing to our local C# namespace, then bind the property using local:AttachedProperties.AnimatedProgress="{Binding Progress}". This code assumes you have a view model for the page with a property called Progress.

You can also set this in code:

MyProgressBar.SetBinding(AttachedProperties.AnimatedProgressProperty,
"Progress");

Once this is wired up we get a lovely animation when we change our progress!

You can find the code for this post on my GitHub Repo.

Missing Xamarin Studio add-ins

Mon, 28 Nov 2016 23:42:58 +0000

In the latest Stable Xamarin Studio (6.1.2) there is a problem with add-ins - the add-in gallery is empty - the add-in endpoint for 6.1.2 is not returning any add-ins.

This means if you are planning on using my Xamarin Studio MvvmCross add-in then you won’t be able to find it.

Luckily there are a number of workarounds:

Use the Alpha channel - on 6.2 all the add-ins reappear.
Add the add-in repository from the previous version by opening the ‘Add-in manager’, going to the ‘Gallery’ tab, selecting ‘Manage Repositories’ from the ‘Repository’ drop-down, tapping ‘Add’ and entering this url - http://addins.monodevelop.com/Stable/Mac/6.1.2/main.mrep
Install the add-in manually by downloading from here and installing it using the ‘Install from file…’ option in the ‘Add-in manager’.

This issue has been raised with Xamarin, so hopefully should be fixed soon.

Xamarin Dev Days!

Sun, 27 Nov 2016 08:43:54 +0000

Our Xamarin Dev Days has been immortalized in a YouTube video! Thanks to Peace of Dreams for filming on the day.

Effects in Xamarin.Forms

Sat, 26 Nov 2016 00:19:40 +0000

Xamarin.Forms is pretty awesome, it provides an abstraction over the UI layer of iOS, Android and UWP apps allowing you to define you UI once either in code (C# or F#), or using XAML - an XML markup language. Write your UI once and it works natively on all devices rendering your UI using the native controls for each platform.

The upside is write-once UIs, the downside is to make this work you only have access to a subset of the APIs available to each UI control so that the Forms code you write will work on all platforms.

Right from the first version of Forms there was a way around this - custom rendered. Forms uses a renderer for each UI control - a native class that takes the Forms element and renders it using a native control. For example if you have a forms Button, the iOS renderer will put a UIKit.UIButton on the screen, the Android renderer will use an Android.Widget.Button. If you wanted to change the way the controls were rendered you could subclass the renderer and add any feature you wanted.

This is good, but is a bit heavyweight if all you want to do is a single property change, and falls down if you want to apply this same property change to multiple elements (you end up writing multiple renderers, one for each element).

Luckily Xamarin Forms has introduced a new technique - effects. This are classes that you can attach to any Xamarin Forms view, and they can manipulate the Forms element or underlying control in any way you want.

To create an effect you have to write a number of classes - a cross-platform RoutingEffect class, and a PlatformEffect for each platform you want to support.

As an example lets create an effect to capitalize the keyboard for an Entry control.

First we create the cross-platform RoutingEffect in our PCL:

namespace Organon.XForms.Effects
{
public class CapitalizeKeyboardEffect : RoutingEffect
{
public CapitalizeKeyboardEffect() :
base("Organon.Effects.CapitalizeKeyboardEffect")
{
}
}
}

This class derives from RoutingEffect, is named using Effect, and passes a string to the base constructor. This string is used to route the effect to the relevant platform specific implementation.

To write the iOS implementation we need to derive from PlatformEffect:

[assembly: ResolutionGroupName("Organon.Effects")]
[assembly: ExportEffect(typeof(CapitalizeKeyboardEffect), nameof(CapitalizeKeyboardEffect))]
namespace Organon.XForms.Effects.iOS.Effects
{
[Preserve(AllMembers = true)]
public class CapitalizeKeyboardEffect : PlatformEffect
{
protected override void OnAttached()
{
var editText = Control as UITextField;
if (editText != null)
editText.AutocapitalizationType = UITextAutocapitalizationType.AllCharacters;
}
protected override void OnDetached()
{
}
}
}

This class has 2 methods we need to implement - OnAttached which is called when the effect is attached to a Forms view, and OnDetached when the effect is detached. You update the native control by accessing the Control property (this is a UIView on iOS and View on Android) and setting it up as needed in OnAttached, then revert your changes in OnDetached. You can also access the Forms element if needed via the Element property. In this example we’re checking that the Control is a UITextField, and if set setting the auto caps to capitalize everything. We could then write something similar in Android.

To use this effect you add it to the Effects collection on your Forms control:

<Entry Placeholder="start typing..." VerticalOptions="Start">
<Entry.Effects>
<effects:CapitalizeKeyboardEffect />
</Entry.Effects>
</Entry>

You can see a couple of effects including this one here:

I’ve been working with a few awesome people on creating an open source library of effects. It’s currently on GitHub under the name ‘OrganonKit’, but is about to be renamed - watch this space for more details plus info on when it will be available as a nuget package.

Xamarin Dev Days Auckland - 19th November 2016

Fri, 04 Nov 2016 07:16:27 +0000

There’s only 1 week to go until the first Xamarin Dev Days in Auckland, hosted by the Auckland Xamarin Meetup on Saturday 19th November. We’ve got an awesome program lined up for this free day and evening, so we thought now would be a good time to share our plans. We have internationally recognized speakers, drinks in the evening and some seriously good prizes to give away!

Location:

Auckland University of Technology, Gov Fitzroy Pl, Auckland, 1010, New Zealand

A big thank you to AUT for sponsoring the room.

Guest speakers

We’ve privileged to have some amazing local and international speakers at this event. As well as listening to the three of us and a local Microsoft technical evangelist you get to hear from:

Michael Ridland

![](AVATAR-small-1.jpg)

Michael Ridland, Xamarin MVP, Community Blogger and the founder of XAM Consulting. A Xamarin fan since the monotouch days and a even bigger Xamarin.Forms fan since it's release. Michael has been leading Xamarin projects years and delivered enterprise apps of the highest quality in the market. Michael (and his team) are the creators of a few Xamarin.Forms components including the FreshMvvm Framework, SlideOverKit and FreshEssentials. You can find him over at michaelridland.com and twitter.com/rid00z.

![](xamconsulting-small-1.png)

Matthew Robbins

![](YWJ3tFCI_400x400-1.jpg)	Matthew is the creator of MFractor - the essential productivity tool for Xamarin.Forms. He's currently a Xamarin developer at JobAdder.
![](logo-horizontal-1-1.png)

Sam Williams

![](hI0WZ08A_400x400-1.jpg)

Sam is a passionate software developer who's obsessed with clean code and F#. In his spare time, you will find him on his VR watching videos or coding some more.

Program for the day:

9:00	Registration and introductions
9:20	Intro to Xamarin
10:00	Xamarin Forms
10:40	Break
10:50	Xamarin Forms performance, power controls and layout systems!
11:20	Xamarin + Azure
12:00	Lunch
1:00	Building Xamarin apps with F#
1:30	Hands on labs/hack time
3:45	Prize giving
5:00	Drinks/snacks at The bluestone room

Refreshments

A big shout out to Datacom for sponsoring the refreshments during the day to keep everyone fueled up. There should be something for everyone but if you have any special dietary needs please let us know as soon as possible.

Hands on labs/hack/Prizes!

The afternoon session is a chance for you to do one of two things. If you are new to Xamarin we have a hands on lab to work through covering Xamarin Forms and Azure. If you have some experience and want to hack on your own thing instead then this will be a good time to do it - you’ll be surrounded by other experts to help with any queries you may have.

In addition EROAD have kindly donated an iPad and a Samsung tablet (luckily not one that explodes) as prizes for the best hack. There’s so many cool features in iOS, Android and UWP, as well as some great toys to play with in Azure so if you can think of something awesome that’s related to transport or health and safety (maybe using Azure cognitive services to see if a driver is angry so shouldn’t drive) you could win big!

There will also be a copy of Jim’s book Xamarin In Action to be given away to one lucky attendee.

Drinks!

Who doesn’t love time to socialize with Xamarin developers? EROAD are sponsoring drinks and snacks at The bluestone room. We’ll be heading there once the day is done so come along for a free drink or two and a chance to talk Xamarin. EROAD is committed to health and safety so if you’re drinking with us, don’t drive home!

Photos

Peace of dreams will be on site to photograph the goings on, a big thanks to them and to Microsoft for sponsoring.

What should you bring?

Bring a laptop with Xamarin installed! There will be WiFi on the day but we can’t guarantee how fast it would be to download and install Xamarin, Android SDK’s etc, so please set everything up before you arrive. We can help if you have any problems but the more prepared you are the better. If you have a Windows laptop and want to build iOS apps then you can sign up with MacInCloud and take advantage of their 24 hour US$0.99 offer.

We look forward to seeing you on the 19th. Our spaces are limited so if you can no longer attend please let us know asap or cancel your registration as we have a lot of interest. If you have any questions about the event please reach out to one of us!

Mohit Singh Baweja - @mohitbaweja Jim Bennett - @jimbobbennett Joshua Fenemore - @im_not_josh

![](Xamarin-logo-copy.png)

Using iOS 10 notifications with iBeacons

Sat, 22 Oct 2016 00:08:10 +0000

I’m a huge fan of iBeacons (and if you’re coming to MS Ignite NZ 2016 then you can hear me talk about them) and they are pretty easy to get started with on iOS. The most basic use case is beacon monitoring - when your app detects a signal from an iBeacon it can wake up and do some processing in the background.

Some of the time this processing might just be showing the user a notification - in a retail example if an app user has looked at a product in your app and they walk into one of your stores you could notify them that the product they looked at was in stock (and direct them right to it using the indoor location capabilities available with iBeacons). This is pretty simple to do - create a location manager, authorize it, monitor for a beacon, detect the event and show a notification. The code below shows the sort of thing you would need to do.

var region = new CLBeaconRegion(proximityId, "My Region");
region.NotifyOnEntry = true;
var locationManager = new CLLocationManager();
locationManager.AuthorizationChanged += (s, e) =>
{
if (e.Status == CLAuthorizationStatus.AuthorizedAlways)
locationManager.StartMonitoring(region);
};
locationManager.RegionEntered += (s, e) => SendNotification();
locationManager.RequestAlwaysAuthorization();

The SendNotification method would then create an iOS local notification and display it.

This is cool, and pretty simple. But thanks to iOS 10 it’s now even easier. As part of iOS 10, Apple introduced a new notifications API, and part of this is a new set of notification triggers. These triggers are classes that trigger a notification - you create a notification request with some content and a trigger that determines when the notification will fire. The simplest example is a trigger that fires after a certain duration (called UNTimeIntervalNotificationTrigger), this will show a notification after a set time interval - and this is similar to the existing capabilities before iOS 10. The trigger we’re interested in though is a new capability - UNLocationNotificationTrigger. This is a trigger that fires when the device enters or leaves a region - be it a geographic region or an iBeacon region.

We can use this new trigger to show a local notification when our app enters or leaves an iBeacon region. First we create the region in exactly the same way as for monitoring - using a proximity UUID and optionally a major and minor to define the region and setting the notify on entry or exit properties as applicable:

var region = new CLBeaconRegion(new NSUuid(ProximityUUID), "MyRegion");
region.NotifyOnEntry = true;
region.NotifyOnExit = false;

Then we ensure the correct permissions are set for notifications and locations:

UNUserNotificationCenter.Current.RequestAuthorization(UNAuthorizationOptions.Alert, (a, e) => { });
var locationManager = new CLLocationManager();
locationManager.RequestWhenInUseAuthorization();

Remembering of course to set the appropriate permission message for location monitoring in the info.plist:

<key>NSLocationWhenInUseUsageDescription</key>
<string>Detect beacons</string>

Next we create the notification:

// Create the content
var content = new UNMutableNotificationContent();
content.Title = "iBeacon notification test";
content.Body = "iBeacon detected";
// Create the location trigger
var trigger = UNLocationNotificationTrigger.CreateTrigger(region, true);
// Create the request
var requestID = Guid.NewGuid().ToString("N");
var request = UNNotificationRequest.FromIdentifier(requestID, content, trigger); UNUserNotificationCenter.Current.RemoveAllPendingNotificationRequests();
UNUserNotificationCenter.Current.AddNotificationRequest(request, e => { });

The notification code is fairly standard iOS 10 boilerplate notification code - we create content for the notification, create a trigger for it, create a notification request based off the content and trigger then add the request to the notification center.

The interesting bit is the trigger - we create a trigger using a static method passing it the region:

UNLocationNotificationTrigger.CreateTrigger(region, true);

The first parameter is our region - in our example this is an iBeacon region but it could also be a location region. The second parameter determines if the notification should be re-scheduled after it has been triggered - set this to true if you want the notification to pop up every time, or false if you don’t want it to repeat.

Once this notification has been registered we can exit our app, and it will wake up as soon as the iBeacon region is entered and show the notification.

Why do it this way?

This may seem like a pointless thing to do - at first glance this doesn’t give us anything more than what we can get by showing the notification using the location managers RegionEntered event. There is one important difference here though.

When using the location manager directly we have to authorize the app always - the user has to agree to have our app monitor their location even when the app is not in use, something that some users don’t like. By using the location based notification triggers we only need to request when in use authorization, not always (you can see this in the code above). Despite only having when in use authorization you app can still detect entering the region even when the app is in the background or even terminated.

Why not to do it this way?

Your app isn’t actually run when the region is entered, all that happens is the local notification is shown. This is fine if that’s all you need (and the new iOS 10 notifications API can allow you to do some pretty neat stuff in a notification), but if you need your app to run then you have to monitor using the location manager and use always authorization.

You can find the example code for this on GitHub.

Xamarin In Action

Tue, 20 Sep 2016 00:55:29 +0000

Over the last few months I’ve been busily writing away on my first book - Xamarin In Action for Manning publications. Manning are a pretty serious publisher, they count Jon Skeet amongst their authors so it was quite an honor to be asked to write for them.

The first 3 chapters are now available to buy under the Manning early access program - under this program you buy the book and get the first 3 chapters now, with more chapters as they are written. You also get to engage with me and give feedback on the book to help make it better for everyone.

Xamarin In Action tries to be different to other book - it’s focused on answering the question of how to build a cross-platform Xamarin mobile app. Unlike other books that just focus on reference material for how to use the various iOS and Android UI widgets and services in C#, this book instead focuses on the main upside of Xamarin - the ability to write cross-platform apps. Unlike just writing for iOS and Android, to make a successful Xamarin app you need to be able to share a large amount of code, but still build your apps in a way that looks and feels like the platform you are targeting. You need to think all the time about what code can be cross-platform, and what code needs to be platform specific.

The best design pattern for doing this is MVVM - you write your models and view models in cross-platform code, then write your views using platform specific iOS or Android code. You can also use cross-platform plugins from your cross-platform code to access device specific features thanks to the such awesome NuGet tricks as bait-and-switch. This book focuses on using MVVM, and specifically MvvmCross, showing you how to build your apps to maximize code re-use and testability.

After showing you how to best build your apps, the book then looks at how to turn them into production quality apps - after an app is only an app once it is on the store. It looks at testing, releasing and monitoring your app once it is released, to allow you to catch bugs early.

You can buy this book from here: Xamarin In Action

More upcoming talks

Sat, 17 Sep 2016 02:51:28 +0000

Ive got a couple of talks coming up around Auckland next week:

Xamarin 102 - a slightly deeper dive - North Shore .Net user group

It’s hard to be a .Net developer and not have heard of Xamarin. It’s now part of Microsoft’s ‘Any developer, any platform’ strategy, and allows you to write C# code that runs on iOS and Android.

In this talk we will briefly introduce Xamarin, then dive right into some code building a cross platform app that loads data from a REST service and shows it on screen - the kind of thing a real world app would want to do. We’ll see how much code can be shared between the iOS and Android app and how much has to be platform specific, look at what design patterns can help to increase the amount of code sharing, and briefly look at Xamarin.Forms as a way of targeting 99% code reuse.

By the time we’re done you should be able to dive right into Xamarin using the free community versions of Visual Studio on Windows or Xamarin Studio on Mac and be ready to play with building your first app!

Details here: http://northshore.netusergroup.org.nz/Meeting/Meetup/?id=M20160922

UI Testing in the real world - Auckland Xamarin Meetup

Testing your mobile apps manually is time consuming and error prone. It’s much better to be able to create automation tests that can reliably, reproducibly and repeatably test your apps.

With Xamarin UITest you can write automation tests in C# that you can run against your apps on simulators or local devices. With Xamarin Test Cloud you can run these tests across thousands of devices in the cloud.

In this talk Jim will give an overview of UITest, show how you can use it to interact with your apps, show how the REPL gives you a way to interact with your app live inside a console.

After this Andrew will pick up the baton and talk about his experiences running these tests in the real world, discussing how to achieve the perfect balance between speed, reliability and ensuring your tests achieve their goal of proving your app works in the hands of actual users.

Details here: https://www.meetup.com/Auckland-Xamarin-Meetup/events/233436741/

Come hear me talk about iBeacons at MS Ignite NZ

Sat, 17 Sep 2016 01:29:51 +0000

I’ll be at Microsoft Ignite New Zealand this year talking about iBeacons, and show how to use them from a Xamarin app. This session will introduce iBeacons and discuss how to use them to build context aware applications, and what types of problems those applications can solve. It will look at how you can notify an app when you are near an iBeacon and how you can use them to track indoor location. It will also discuss the downsides, such as security concerns and problems around beacon deployments. As well as discussing the concepts it will also contain a live coding Xamarin demo showing just how easy it is to beacon-enable your apps, with live demos of notifications and indoor location tracking.

MS Ignite runs from 25-28 October at the SkyCity convention centre in Auckland, New Zealand.

Update - the video of this talk is now on Channel 9:

https://channel9.msdn.com/Events/Ignite/New-Zealand-2016/M324

More on Xamarin Studio add-ins

Wed, 27 Jul 2016 08:48:05 +0000

As a follow up to my previous post about creating an add-in for Xamarin Studio I thought I’d go into the details of the project template XML.

When creating an add-in that adds a new project type you define each project as a Project node inside the xpt.xml file. This project node has some attributes, and sub nodes made up of options, references, files and packages.

Think of this post as a sort of API reference for these based on what I’ve discovered, or been taught. Once again this post would not be possible without the help of Mikayla Hutchinson - she is awesome.

Variables

In this file you can use ${ProjectName} to refer to the name of the solution. This is not the file name, just the solution name, so MySolution.sln would be called MySolution and every instance of ${ProjectName} would be replaced with MySolution.

The `Project` node

<Project name = "${ProjectName}.Core" directory = "./${ProjectName}.Core" type="C#PortableLibrary">

The name is the name of the .csproj file. In the files for this project the ${ProjectName} is set to this name.

The directory is the subdirectory under the solution where this project is created.

The type is the type of the project. The 3 types I know about are:

C#PortableLibrary - PCL
MonoDroid - Android app
XamarinIOS - iOS app

The `Options` node

This is a sub-node of Project, and is used to set options on the project file. The attributes I am aware of are:

PCL projects:

<Options Target="Library" TargetProfile = "Profile259"/>

Target="Library" - sets this project as a library
TargetProfile="Profile259" - sets the target profile

Android projects:

<Options AndroidApplication="true" AndroidResgenFile="Resources\Resource.Designer.cs" GenerateSerializationAssemblies="Off" ProductVersion="8.0.30703" SchemaVersion="2.0" AppDesignerFolder="Properties" FileAlignment="512" JavaMaximumHeapSize="1G"/>

AndroidApplication="true" - Marks this project as an Android app
AndroidResgenFile="Resources\Resource.Designer.cs" - Tells the Android app which file is autogenerated from the resources
JavaMaximumHeapSize="1G" - Sets the Java heap size, 1G is better than the default 512M

Not sure exactly what the other attributes do - if you know please tell me and I’ll update this.

iOS projects:

<Options />

I don’t know any of the options here, again if you know please tell me!

The `References` node

This is where you define references to libraries or other projects. No need to add references to libraries that come from NuGet here, as these are set via the Packages node.

<References>
<Reference type="Package" refto="Mono.Android.Export" />
<Reference type="Project" refto="${ProjectName}.Core" />
</References>

type - Package for an assembly either from the .Net framework or from a library that you are shipping, Project for a reference to another project in your solution
refto - the assembly to reference (for framework assemblies), or the project name for other projects in the solution.
HintPath="xxx" - specifies the hint path for references to libraries outside of the framework (such as one you are shipping with the add-in)

The `Files` node

This is where you define the actual files to copy to your project. These files must come from somewhere, so you need to add them to your project with Copy to output directory ticked. You also need to add these files to your Manifest.addin.xml file as Import entries in the Runtime node.

<Files>
<File name="Properties/AndroidManifest.xml" src="Droid/Properties/AndroidManifest.xml"/>
<RawFile name="Resources/drawable/icon.png" src="Droid/Resources/drawable/icon.png"/>
<File name="Pages/FirstPage.xaml" BuildAction="EmbeddedResource" src="Core/Pages/FirstPage.xaml"/>
<File name="Pages/FirstPage.xaml.cs" AddStandardHeader="True" src="Core/Pages/FirstPage.xaml.cs" DependsOn="FirstPage.xaml" />
</Files>

There are two possible nodes here, File and RawFile. Use File for text files as this will do file ending/unicode conversion. Use RawFile for binary files, such as images or libraries to avoid any conversion.

name - the file name including the directory for the file to write to
src - the file name and path of the source file inside your bundle
BuildAction - the build action for this file. For normal compile you don’t need this but for Xamarin.Forms Xaml files you need to set this to EmbeddedResource .
AddStandardHeader="True" - don’t know what this does, but is used for xaml files for forms.
DependsOn - sets the file that this depends on, so used for specifying the Xaml file for a xaml.cs code behind file.

The `Packages` node

Here you can define what NuGet packages to install in your app. These are installed as if you installed them manually, so all dependent packages are installed.

<Packages>
<Package Id="Xamarin.Forms" Version="2.3.0.107"/>
<Package id="MvvmCross" version="4.2.2"/>
<Package id="MvvmCross.Forms.Presenter" version="4.1.4"/>
</Packages>

id - The Id of the NuGet package to install
version - The specific version to install if needed

This doc is nowhere near complete, so if you know of other things to add please let me know and I’ll update it.

Creating an add-in for Xamarin Studio

Fri, 01 Jul 2016 23:56:53 +0000

All good IDE’s come with a framework for extending them, and Xamarin Studio is no different. It is built on top of MonoDevelop which makes extensive use of add-ins to provide a lot of it’s features, and it’s really easy to add your own.

Recently I created my own add-in to help out when building MvvmCross projects. If you wanted to do a ‘File -> New Solution’ in Xamarin Studio and create a simple, working MvvmCross cross platform app targeting iOS and Android you had to do a few things to make it work - you had to create the solution, ensure you have a PCL core project, an iOS app and an Android app, manually add the nuget packages for MvvmCross and either write the code yourself, or install the MvvmCross starter pack nuget package. This starter was helpful, it contained sample code and instructions showing what you needed to change in the code files that were already created in your apps. As great as this is though, I thought it might be better if you could do a ‘File -> New Solution’ and from the New Project dialog choose to create a fully working MvvmCross iOS and Android sample app to save a couple of minutes. #FirstWorldProblems.

Lets now walk through just how I did it and see how easy it was to create my add-in. I’ll be referring to my add-in in the examples, and you can find the code on my github.

You can use Xamarin Studio to write add-ins, but first you need to install an add-in! Select ‘Xamarin Studio -> Add Ins…’ and add the ‘Add-in Maker’ add-in.

Once this is added I’d recommend restarting Xamarin Studio. If you don’t restart after adding add-ins they don’t always work correctly - new project types can be missing or in the wrong place.

To create your add-in do ‘File -> New Solution’ and select ‘Xamarin Studio AddIn’ from the ‘Other -> Miscellaneous’ section.

When you create the add-in project you get a few things set up for you. Firstly you get a package reference to MonoDevelop.AddIns which is the NuGet package that provides a lot of the features you need. Secondly you get a couple of extra files in the Properties folder:

AddInInfo.cs - This file allows you to provide information about your add-in such as the name, category (so where it appears in the Add-In manager), description, author, version etc.
Manifest.addin.xml - This manifest file defines what your add-in actually does. It lists any files or resources that are shipped inside the add-in, and defines what features are available.

My MvvmCross add-in provides a new solution type containing 3 projects. Adding a new solution type is actually pretty easy - you create an xml file that defines what projects go into the solution, with each project defined inside that xml file, and the files inside the project are provided by you as raw files that get copied in.

So if you look inside the Templates folder in the add-in project you will see 3 folders and an xml file. The MvvmCross.Solution.xpt.xml file is the one that defines the projects. If you open this file and browse the contents you will see the following sections:

TemplateConfiguration - This section defines the configuration for this solution template, such as it’s name, description, icon, programming language and where in the New Project dialog it should be.
Combine - This section is the actual ‘instructions’ to to create the solution. This section has 3 Project nodes in it, and these represent the actual projects that will be created in the solution. In the project node we define the project name, type, directory that it lives in. In the sub-nodes we define the options that are set on the project, as well as contents of the project such as references and the files to include. These files that are included point to an actual file inside the project - and it’s this that makes it easy to create such an add-in. What you do is create all the files yourself, so create a project that matches the output of your add-in and add these files to your add-in project without a build action but copied to output. When your add-in runs it will build the project based on the instructions given, copying the files you’ve added to the new project.

One thing to note is the variables that are used such as ${ProjectName}. There are a few of these. and these are replaced with appropriate values when your add-in is run. For example ${ProjectName} is replaced with the name you put into the ‘Project Name’ in the ‘New Project’ dialog. As well as putting these inside the xml file you also need to use these in the source files that are shipped with the add-in. You also have to be aware that these variables can change based on the scope - so for example is you created a new solution using the add-in called ‘Foo’, when building the solution ${ProjectName} would be ‘Foo’, but if the solution creates a project call ‘Foo.Core’, inside any code files for that project ${ProjectName} would actually be ‘Foo.Core’, and to get ‘Foo’ you have to use ${SolutionName}.

Have a look at the code in the MvvmCross add-in and you’ll see this in action. It’s actually pretty self-explanatory once you start reading a real example, which is one of the great things about the Xamarin Studio add-ins.

Once you have built your add-in, you can run it and it will launch a new instance of Xamarin Studio with your add-in installed for you to test out. If you want to ship the add-in to other people to try out you need to package it using mdtool, which is buried inside the Xamarin Studio application package:

/Applications/Xamarin\ Studio.app/Contents/MacOS/mdtool setup pack SomeAddin.dll

This will spit out an mpack file - a mono package that contains the add-in. You can send this file out and users can add it from the ‘AddIn Manager’ using the ‘Install from file…’ option.

The last file to note is `addin-project.xml’. You need to create one of these if you are want to ship the add-in to the gallery at http://addins.monodevelop.com. When you ship an add-in it will grab code from your repo and compile it up, and this xml file is used to tell their compiler what to do.

Easy, and awesome!

Now go off and write an add-in, and ping me here or on twitter if you have any questions.

This blog post would be complete without a shoutout to Mikayla Hutchinson and Lluis Sanchez Gual from Xamarin for being awesome and helping me get my add-in working and up on the add-ins gallery. You rock!

MvvmCross add-ins for Visual Studio and Xamarin Studio

Thu, 26 May 2016 09:22:27 +0000

I’ve been using MvvmCross a lot at work and it’s pretty cool. I’ve got a big upcoming personal project that I’ve started working on (more info soon I promise - it’s something pretty awesome), and I’ve decided to use MvvmCross for it. Unfortunately despite all it’s awesomeness, creating a boilerplate cross platform solution using MvvmCross is still a bit of work. There is the starter pack nuget, but using this still means a bit of work copying code, setting up projects and remembering to remove the starter pack once you are finished just from your packages.config (not from your installed packages or you lose a load of files).

Ideally I’d like an add-in for Visual Studio and Xamarin Studio where you can create a new solution and it creates all the projects you need for a cross platform app - so core/iOS/Droid/UWP/Wpf/Win8 on Visual Studio and core/iOS/Droid/Mac on Xamarin Studio. There is Ninja Coder but this is Visual Studio only and is overkill for my needs as it does pretty much everything you could want, all I want is a consistent way of creating projects using Visual Studio and Xamarin Studio that match the starter pack.

I had a chat with Tomasz Cielecki (or CheeseBaron as you may know him), and we talked about creating an add-in that does just this. So over the past few evenings I’ve been teaching myself how to create add-ins for Visual Studio and Xamarin Studio and I’ve finally managed to get it working.

The Visual Studio one is available here. Grab the source if you want a nose at how it works, or grab the release to try it out. Once I’ve had some feedback I’ll put it up on the Visual Studio gallery.

The Xamarin Studio version is available here. Again, source is available and a release build is there you can try out. I’ve also submitted it to the alpha channel for Xamarin Studio add-ins for v6 and above, so once it’s accepted you should see it in the add-in gallery in the latest beta of Xamarin Studio. This one was hard to do as there is minimal documentation, so a big shout out to Mikayla Hutchinson for generally being awesome and answering my dumb questions.

I’ll blog in more detail about how I created these add-ins at a later date. For now though please grab them and try them out. I’d appreciate as much feedback as possible!

Update!

These add-ins are now live in the Xamarin Studio stable add-in repository and on the Visual Studio Gallery. This means you can install them from inside Xamarin Studio’s add-in manager and Visual Studio’s extensions and updates. The Xamarin Studio add-in needs XS 6 so you’ll need the latest Stable or Beta, the Visual Studio extension needs VS2015.

Fixing issues with Multidex on pre-lollipop devices on Windows

Fri, 20 May 2016 08:20:16 +0000

When building large apps on Android you may end up getting a weird ‘Java has exited with code 2’. This is caused by a limitation when building Android apps of 65,536 fields. Sounds odd I know, but your app can’t contain are than 65k fields - a field being a method, property or similar. You may think that this is a lot, but this also includes any other libraries you use, and just by including google maps you’re already half way there. You can get more information on this on the Android developer page.

Luckily there is a way round this - multidex. When Android compiles your app it puts the code into a dex file and these dex files have the 64k limit. To exceed this limit the compiler can create multiple dex files, each with 64k fields in it. This seems like an overcomplicated solution, but from our perspective it’s simple - just tick a box on the project settings and the Xamarin SDK/Android SDK does the magic.

Your Android app needs an application class to run, and if you don’t have a custom application class in your app it gets the default Android one - Application. When you set your app to use multidex the default application class that is used is MultiDexApplication. Now if you have your own custom application class that derives from Application, then setting multidex will tweak your code so that it derives from MultiDexApplication. In addition to this when your app is launched it needs to run the application. In the case of an app with multiple dex files it needs to look in all the dexes to find the application class. On Lollipop and above this is not a problem, the OS is built to understand this and teach for the application. However on previous versions such as KitKat multidex wasn’t built in, so the OS will only ever look in the first dex file as this is all it knows about. If the application class is not found, the app crashes on startup. Again normally this is nothing to worry about. When your app is built it knows which classes to put in the first dex file - it creates a list of classes to put in the main dex and uses this during the build. This means when your app is launched on a pre-lollipop device the right code is in the main dex so your app is launched, and because it uses MultiDexApplication this will cause the other dexes to load.

So for a developer - just tick the box and job done.

Except when it goes wrong. I’ve spent the last few weeks with an issue where my large app crashes instantly on KitKat because it can’t find the application class. Luckily I’ve had access to Xamarin support to help look into this.

It seems the issue is caused by a bug in the Android SDK on Windows. See https://code.google.com/p/android/issues/detail?id=187721 for more details, but basically when it generates the list of classes to put in the first dex, the list ends up empty.

This only affects the Android SDK on Windows, and so far it’s in the latest build tools versions - 23 and up. There is a simple fix, a big thanks to Jon Douglas at Xamarin for providing this.

What you need to do is edit the mainDexClasses.bat file inside android-sdk\build-tools\23.0.3 (or whatever is your latest version). It also helps to delete all earlier versions to ensure the wrong one doesn’t get used. In this file, near the bottom, change:

if DEFINED output goto redirect
call "%java_exe%" -Djava.ext.dirs="%frameworkdir%" com.android.multidex.MainDexListBuilder "%disableKeepAnnotated%" "%tmpJar%" "%params%"
goto afterClassReferenceListBuilder
:redirect
call "%java_exe%" -Djava.ext.dirs="%frameworkdir%" com.android.multidex.MainDexListBuilder "%disableKeepAnnotated%" "%tmpJar%" "%params%" 1>"%output%"
:afterClassReferenceListBuilder

to:

SET params=%params:'=%
if DEFINED output goto redirect
call "%java_exe%" -Djava.ext.dirs="%frameworkdir%" com.android.multidex.MainDexListBuilder %disableKeepAnnotated% "%tmpJar%" %params%
goto afterClassReferenceListBuilder
:redirect
call "%java_exe%" -Djava.ext.dirs="%frameworkdir%" com.android.multidex.MainDexListBuilder %disableKeepAnnotated% "%tmpJar%" %params% 1>"%output%"
:afterClassReferenceListBuilder

And that’s it! Rebuild your app and it should now run on pre-lollipop devices.

This only affects Android builds on Windows, not on Mac. Also be aware though that there is a different bug in Xamarin on the alpha and beta channel that causes issues with Multidex, so if you’re running into a problem on these versions you’ll need to update to stable until a fix is released.

And here’s a better blog post on it from the engineer at Xamarin who helped me solve the issue.

http://www.jon-douglas.com/2016/09/05/xamarin-android-multidex/

Xamarin Evolve 2016 roundup

Mon, 16 May 2016 04:27:20 +0000

As part of my work as a mobile developer at EROAD I blogged about my experiences at Xamarin Evolve this year. You can read it on LinkedIn

My talk from Xamarin Evolve 2016 is now on YouTube

Wed, 11 May 2016 08:03:22 +0000

My talk from Xamarin Evolve 2016 is now up on YouTube!

Contextual awareness is a fast-growing trend in mobile development. In this session, Jim will take you through a quick overview of beacon technology, including Apple's iBeacons and Google's Eddystone, before diving into some code. We'll even build a simple app live on stage to demonstrate how you can detect and wake up beacons, as well as detect user proximity to beacons.

Building a Xamarin Android app - part 6

Thu, 24 Mar 2016 08:27:44 +0000

This is the sixth part in the my series about building an Android app using Xamarin.Android. I highly recommend reading the rest of the series first. The previous parts are:

Editing and deleting our counters

So far we have a pretty nice but simple app - we can add new counters and increment them. The next thing we need to add is the ability to edit or delete a counter. Lets start at the bottom of our stack and work up to the UI. We need to be able to edit or delete the counter - so we need a way to update the record or delete the row in out SQLite db, we need commands on our ViewModel to expose this, and we need a nice UI to handle the user interaction.

Basic UI design

Before we begin though, we should consider how the user will delete a counter, so that we can implement it in the correct way. Our recycler view showing the counters is our way to interact with them - tapping on the plus button will increment the counter, so it makes sense to use this to drive editing a counter. We already make use of tapping on the plus button, so a simple tap on the recycler view item may not be the best way to drive the edit (it would be too easy to tap the wrong thing and edit instead of incrementing the counter). Instead we can take advantage of one of Androids other UI paradigm - the long touch. If a user performs a long touch on the counter in our recycler view we can show a new activity where they can edit the name or description of the counter, or delete it (and we can re-use this screen later to show the counters history).

Lets crack on with the implementation.

Database

In our DatabaseHelper we already have the ability to update counters using the AddOrUpdateCounterAsync method, so all we need to do is add an async delete method that takes a counter and deletes the row from SQLite. This needs to be added to the interface:

public interface IDatabaseHelper
{
Task DeleteCounterAsync(Counter counter);
...
}

And the implementation:

public async Task DeleteCounterAsync(Counter counter)
{
var connection = new SQLiteAsyncConnection(_dbPath);
await connection.DeleteAsync(counter);
OnCountersChanged();
}

After the row is deleted we also need to raise the CountersChanged event to notify anything that is interested, such as our CountersViewModel that the counters have changed.

And obviously as we are good developers we need some tests for this in our Android unit test suite in the DatabaseHelperTests fixture to cover the deleting of rows and raising the collection changed event:

[Test]
public void DeletingACounterShouldDeleteTheCounter()
{
var dbfile = Path.Combine(RootPath, Guid.NewGuid().ToString("N") + ".db3");
DatabaseHelper.CreateDatabase(dbfile);
var db = new DatabaseHelper();
var counter = new Counter
{
Name = "TestCounter",
Description = "A test counter"
};
var res = Task.Run(async () =>
{
await db.AddOrUpdateCounterAsync(counter);
return 0;
}).Result;
var counters = Task.Run(async () => await db.GetAllCountersAsync()).Result;
counters.Should().HaveCount(1);
res = Task.Run(async () =>
{
await db.DeleteCounterAsync(counter);
return 0;
}).Result;
counters = Task.Run(async () => await db.GetAllCountersAsync()).Result;
counters.Should().HaveCount(0);
}
[Test]
public void DeletingACounterShouldRaiseTheCollectionChangedEvent()
{
var dbfile = Path.Combine(RootPath, Guid.NewGuid().ToString("N") + ".db3");
DatabaseHelper.CreateDatabase(dbfile);
var db = new DatabaseHelper();
var counter = new Counter
{
Name = "TestCounter",
Description = "A test counter"
};
var res = Task.Run(async () =>
{
await db.AddOrUpdateCounterAsync(counter);
return 0;
}).Result;
var eventRecorder = new EventRecorder(db, nameof(DatabaseHelper.CountersChanged));
eventRecorder.RecordEvent();
res = Task.Run(async () =>
{
await db.DeleteCounterAsync(counter);
return 0;
}).Result;
eventRecorder.Should().HaveCount(1);
}

ViewModel

What we need here is a ViewModel that exposes properties for the name and description of our counter, as well as providing a mechanism to save or delete it. We could build a new ViewModel for this, but we already have 2 ViewModels that represent counters with various levels of functionality on them - a NewCounterViewModel that allows the setting of the Name and Description properties, as well as a command to save the new counter to the database, and a CounterViewModel that has a readonly view of the counters details with a command to increment it’s value. Ideally we should follow the DRY principle and only have one, so instead of adding a new one, lets take this opportunity of refactoring the code to have only one ViewModel, and add our new requirements to it.

So lets take the CounterViewModel and make this work for all our scenarios. First we need to change the Name and Description properties. These are currently read only and just return the value from the counter. What we’ll do with these is make them read/write properties with a property change notification and a backing field, and set them in the constructor. We don’t want to use passthrough properties, we need a backing field. This is because we want to allow the user to cancel their changes when they edit, if we pass the value straight through to the counter when we set it we will have to store the original value somewhere to allow us to revert it. It’s easier to save the values in a field and only apply them to the counter if the user saves the changes.

public CounterViewModel(Counter counter, IDatabaseHelper databaseHelper)
{
...
Name = counter.Name;
Description = counter.Description;
}
private string _name;
public string Name
{
get { return _name; }
set { Set(ref _name, value); }
}
private string _description;
public string Description
{
get { return _description; }
set { Set(ref _description, value); }
}

The NewCounterViewModel also has commands to add a new counter and navigate back. The call on the database helper to add the new counter is the same as the one we will need to use to save the edits, AddOrUpdateCounterAsync, so lets copy the AddCounterCommand from the NewCounterViewModel and rename it to SaveCommand so it can be used for both new and edits. We’ll also need to add the IDialogService and INavigationService to the constructor parameters so they can be used by this command.

public CounterViewModel(Counter counter, IDatabaseHelper databaseHelper, IDialogService dialogService, INavigationService navigationService)
{
_dialogService = dialogService;
_navigationService = navigationService;
...
}
private RelayCommand _saveCommand;
public RelayCommand SaveCommand => _saveCommand ?? (_saveCommand = new RelayCommand(async () => await SaveAsync()));
private async Task SaveAsync()
{
if (string.IsNullOrEmpty(Name))
{
await _dialogService.ShowError("The name must be set", "No name", "OK", null);
return;
}
if (string.IsNullOrEmpty(Description))
{
await _dialogService.ShowError("The description must be set", "No description", "OK", null);
return;
}
_counter.Name = Name;
_counter.Description = Description;
await _databaseHelper.AddOrUpdateCounterAsync(_counter);
_navigationService.GoBack();
}

There is one code change made here as well - in the NewCounterViewModel a new Counter was created with the values from the properties. Here we use the _counter field, and update the values from our properties.

We can then add the GoBackCommand as well.

private RelayCommand _goBackCommand;
public RelayCommand GoBackCommand => _goBackCommand ?? (_goBackCommand = new RelayCommand(() => _navigationService.GoBack()));

Next we need to add the DeleteCommand to expose the new delete functionality we’ve added to the database helper.

private RelayCommand _deleteCommand;
public RelayCommand DeleteCommand => _deleteCommand ?? (_deleteCommand = new RelayCommand(async () => await DeleteAsync()));
private async Task DeleteAsync()
{
if (await _dialogService.ShowMessage($"Are you sure you want to delete {Name}?", "Delete counter", "Yes", "No", null))
{
await _databaseHelper.DeleteCounterAsync(_counter);
_navigationService.GoBack();
}
}

Nothing unexpected or new here - we show a dialog confirming with the user that they want to delete, and if they say yes delete the counter from the database and navigate back.

To test this we also need to copy the tests from the NewCounterViewModelTests fixture, making the relevant changes to change the type of view model that is constructed. We can then also add some tests for the DeleteCommand, as well as ensuring the saving tests will copy the values to the counter.

[Test]
public void ExecutingTheSaveCommandUpdatesTheChangesOnTheCounter()
{
var counter = new Counter {Value = 10, Name = "Name", Description = "Description"};
var vm = new CounterViewModel(counter, _mockDatabaseHelper.Object, _mockDialogService.Object, _mockNavigationService.Object);
vm.Name = "New Name";
vm.Description = "New Description";
vm.SaveCommand.Execute(null);
counter.Name.Should().Be("New Name");
counter.Description.Should().Be("New Description");
}
[Test]
public void ExecutingTheSaveCommandUpdatesTheCounterInTheDatabase()
{
var counter = new Counter { Value = 10, Name = "Name", Description = "Description" };
var vm = new CounterViewModel(counter, _mockDatabaseHelper.Object, _mockDialogService.Object, _mockNavigationService.Object);
vm.Name = "New Name";
vm.Description = "New Description";
vm.SaveCommand.Execute(null);
_mockDatabaseHelper.Verify(d => d.AddOrUpdateCounterAsync(counter));
}
[Test]
public void ExecutingTheSaveCommandNavigatesBack()
{
var vm = CreateCounterViewModel();
vm.SaveCommand.Execute(null);
_mockNavigationService.Verify(n => n.GoBack());
}
[Test]
public void ExecutingTheDeleteCommandConfirmsTheDelete()
{
var vm = CreateCounterViewModel();
vm.DeleteCommand.Execute(null);
_mockDialogService.Verify(d => d.ShowMessage($"Are you sure you want to delete {vm.Name}?", "Delete counter", "Yes", "No", null));
}
[Test]
public void ExecutingTheDeleteCommandAndSelectingYesOnTheDialogDeletesTheCounter()
{
var counter = new Counter { Value = 10, Name = "Name", Description = "Description" };
var vm = new CounterViewModel(counter, _mockDatabaseHelper.Object, _mockDialogService.Object, _mockNavigationService.Object);
_mockDialogService.Setup(d => d.ShowMessage($"Are you sure you want to delete {vm.Name}?", "Delete counter", "Yes", "No", null))
.ReturnsAsync(true);
vm.DeleteCommand.Execute(null);
_mockDatabaseHelper.Verify(d => d.DeleteCounterAsync(counter), Times.Once);
}
[Test]
public void ExecutingTheDeleteCommandAndSelectingNoOnTheDialogDoesNotDeleteTheCounter()
{
var counter = new Counter { Value = 10, Name = "Name", Description = "Description" };
var vm = new CounterViewModel(counter, _mockDatabaseHelper.Object, _mockDialogService.Object, _mockNavigationService.Object);
_mockDialogService.Setup(d => d.ShowMessage($"Are you sure you want to delete {vm.Name}?", "Delete counter", "Yes", "No", null))
.ReturnsAsync(false);
vm.DeleteCommand.Execute(null);
_mockDatabaseHelper.Verify(d => d.DeleteCounterAsync(counter), Times.Never);
}

After doing this we can tidy up by deleting the NewCounterViewModel and NewCounterViewModelTests classes. We also need to make the new code work. At the moment the NewCounterActivity uses a NewCounterViewModel, and gets it from the ViewModelLocator, clearing the values each time. We can delete it from the ViewModelLocator, change the type of the ViewModel property on the activity to be CounterViewModel, change the command that gets called to save, and create a new one with a new counter.

public CounterViewModel ViewModel { get; private set; }
protected override void OnCreate(Bundle savedInstanceState)
{
...
ViewModel = GetCounterViewModel();
...
}
protected virtual CounterViewModel GetCounterViewModel()
{
return new CounterViewModel(new Counter(),
ViewModelLocator.DatabaseHelper,
ViewModelLocator.DialogService,
ViewModelLocator.NavigationService);
}
public override bool OnOptionsItemSelected(IMenuItem item)
{
switch (item.ItemId)
{
...
case Resource.Id.action_save_counter:
ViewModel.SaveCommand.Execute(null);
return true;
...
}
}

Notice how the GetCounterViewModel is protected and virtual? We’ll see why later when we look at creating the views.

We then need to update the CountersViewModel to pass the new parameters to the constructor for our CounterViewModel. These extra parameters need to be added to the constructor so that the IoC container can populate them, then passed through once the CounterViewModels are created.

public CountersViewModel(IDatabaseHelper databaseHelper, INavigationService navigationService, IDialogService dialogService)
{
_navigationService = navigationService;
_dialogService = dialogService;
...
}
public async Task LoadCountersAsync()
{
...
foreach (var counter in counters)
_counters.Add(new CounterViewModel(counter, _databaseHelper, _dialogService, _navigationService));
}

Also here the CountersViewModelTests should be updated to mock up these extra parameters and pass them through when constructing the CounterViewModels.

We’re almost there with the ViewModels part. The last thing to think about is how we are actually going to start the editing process. Android has a UI paradigm of a long press on an item in a list that leads to extra functionality, such as editing. We can use this here - the user can long press on the counters in our recycler view and we can open an edit activity. To make this work we need to add a command to our CounterViewModel that using the INavigationService to navigate to a new screen, passing the counter through. First we define a new key for this in our ViewModelLocator.

public const string EditCounterPageKey = "EditCounterPage";

Then we add the command to our CounterViewModel.

private RelayCommand _editCommand;
public RelayCommand EditCommand => _editCommand ?? (_editCommand = new RelayCommand( Edit));
private void Edit()
{
_navigationService.NavigateTo(ViewModelLocator.EditCounterPageKey, _counter);
}

Then we add a test to the CounterViewModelTests.

[Test]
public void ExecutingTheEditCommandNavigatesToTheEditScreen()
{
var counter = new Counter();
var vm = new CounterViewModel(counter, _mockDatabaseHelper.Object, _mockDialogService.Object, _mockNavigationService.Object);
vm.EditCommand.Execute(null);
_mockNavigationService.Verify(n => n.NavigateTo(ViewModelLocator.EditCounterPageKey, counter));
}

Done. We now have everything in our ViewModels ready to start on the views.

Views

We need to do a few things with our views to get editing working - we need to handle the long press on the recycler view to launch the editing, then create a new activity and layout for the editing screen.

Lets start with the long press. This is easy enough to implement in our existing CounterViewHolder class - we just add an event handler and execute our command from there.

public CounterViewHolder(View itemView) : base(itemView)
{
..
itemView.LongClick += ItemLongClick;
}
private void ItemLongClick(object sender, View.LongClickEventArgs e)
{
_counterViewModel.EditCommand.Execute(null);
}

Android also supports a nice animation for this - tap and see a highlight starting at the touch point and spreading out.

This is easy enough to implement, we just need to change the foreground of the CardView in the counter_view.axml layout file to use a stock Android selectable item background.

<android.support.v7.widget.CardView
...
android:foreground="?android:attr/selectableItemBackground">

We’ve got the long click looking awesome, and we’ve got the command executing. Now lest create an activity that the command can launch. For this we can create EditCounterActivity, and instead of deriving from BaseActivity like all the others, instead we can derive from NewCounterActivity. This activity already has most of what we need - it has a layout that shows the name and description in edit controls which we can reuse, it has a save button, it uses the right ViewModel type. All we need to change when we override it is how it creates the ViewModel, and add an extra menu item for the delete button.

Remember earlier we created a protected virtual method in the NewCounterActvity that creates a new CounterViewModel? Well the reason for creating it as a virtual is that we can override it in our EditCounterViewModel and create a new one wrapping the counter passed to our navigation call in the EditCommand of the CounterViewModel.

protected override CounterViewModel GetCounterViewModel()
{
var navigationService = ViewModelLocator.NavigationService;
var counter = (Counter) ((AppCompatNavigationService) navigationService).GetAndRemoveParameter(Intent);
return new CounterViewModel(counter,
ViewModelLocator.DatabaseHelper,
ViewModelLocator.DialogService,
navigationService);
}

We have to cate the INavigationService to a AppCompatNavigationService to get data out of it. This is because data passing is very platform specific, we need to give it the Intent that was used to launch the activity so that it can get the data out, and this is Android specific. The interface is designed to be portable so contains no platform specific code, so we cast to the Android variant to get the data out.

Next we need the delete button on the menu. For this we need a new menu called edit_counter_menu.xml containing this button.

<?xml version="1.0" encoding="utf-8" ?>
<menu xmlns:android="http://schemas.android.com/apk/res/android"
xmlns:app="http://schemas.android.com/apk/res-auto">
<item android:id="@+id/action_delete_counter"
app:showAsAction="always"
android:icon="@drawable/ic_delete_white_48dp"/>
</menu>

Th image here, ic_delete_white_48dp is once again dowloaded from Google’s material icons and dropped into all the drawable folders.

In our activity we now need to create the menu, and handle when it gets pressed.

public override bool OnCreateOptionsMenu(IMenu menu)
{
base.OnCreateOptionsMenu(menu);
Toolbar.InflateMenu(Resource.Menu.edit_counter_menu);
return true;
}
public override bool OnOptionsItemSelected(IMenuItem item)
{
switch (item.ItemId)
{
case Resource.Id.action_delete_counter:
ViewModel.DeleteCommand.Execute(null);
return true;
default:
return base.OnOptionsItemSelected(item);
}
}

These work the same way as the NewCounterActivity described in an earlier post.. The base calls in both these methods call down to the NewCounterActivity so that the original menu with the save button is created and handled.

The last little but we need to do is a UI tweak. The NewCounterActivity sets the title to New Counter, but it would look nicer if we show that we are editing a counter. Also by default Android will set the focus on the first edit view, where MVVMLight sets the text, the focus ends up at the start of the text instead of the end. We can get round this by calling SetSelection on the Name edit view in our OnCreate method after we bind the data in the base OnCreate method.

protected override void OnCreate(Bundle savedInstanceState)
{
base.OnCreate(savedInstanceState);
Title = $"Edit {ViewModel.Name}";
Name.SetSelection(Name.Text.Length);
}

The last thing we need to do is wire up this activity to the new key we created for navigation in the ViewModelLocator. All our wire ups are done in the MainActivity so lets add another entry.

public MainActivity()
{
...
navigationService.Configure(ViewModelLocator.EditCounterPageKey, typeof(EditCounterActivity));
}

Done!

Lets try it all out

We have our database code written, we have our ViewModels working and our UI wired up. Lets try it out.

The code for this can be found in GitHub on the Part6 branch at https://github.com/jimbobbennett/StupendousCounter/tree/Part6

In the next part we’ll look at showing the history information that we’re storing against each counter.

Installing Google Apps in a Marshmallow emulator

Tue, 01 Mar 2016 08:49:37 +0000

I’ve been trying to play with the Google Nearby Message API recently as this has capabilities to talk to Eddystone beacons. The problem I’ve come up against is that the example code uses the new Marshmallow permissions so I’m having a bit of a headache getting it working.

I’ve got a Tesco Hudl 2 as my only physical device, and thanks to the annoying way Android is repackaged by each hardware provider I can’t update it past Lollipop.

The other option is an emulator, but these need Google Play services to enable the Nearby APIs.

You’d think this would be easy, it’s a pretty normal thing to do. Unfortunately it’s not so easy. All the emulators don’t come with these by default, which seems a strange oversight. Instead you need to install them later. Again, you’d expect this to be simple. But it’s not - at least not for Marshmallow. It’s taken about 6 hours of downloading, trying and googling to get it working.

So here’s how. All links mentioned are working at the time of writing.

First, install GenyMotion. Normally I would be extolling the virtues of Xamarin Android Player, but in this case I could not get their Marshmallow preview to work by following the Xamarin instructions.

Next create a new virtual device - I’ve been using the Google Nexus 5X - 6.0.0 - API 23 - 1080x1920 image. Start this up.

Then you need to install an ARM translator, this is available here. Download it, drag it on to the running emulator. A dialog will pop up asking if you want to flash the ROM, click yes and let it install. Once done reboot the device from the ADB command prompt using the command:

adb reboot

Then close and re-open the emulator.

After the translator you need to install the Lollipop version of the Google Apps package from here. Yup - the Lollipop version. Same as for the translator, drag it on, click yes, then reboot using ADB once done.

Once the Google Apps are installed, from Settings -> Accounts log in with your Google account.

Now update to the Google Apps for Marshmallow, which is available here. Install, reboot as before. It’s important that you log in before installing the Marshmallow version, otherwise logging in just won’t work.

Now you can launch the Google Play Store and install and update apps as required.

Done - An overly complicated solution to what should be a simple thing.

Update

A much better description with proper credit to everyone involved is up here, courtesy of @cheesebaron:

http://blog.ostebaronen.dk/2016/04/installing-gapps-in-visual-studio.html

Building a Xamarin Android app - part 5

Thu, 18 Feb 2016 08:59:14 +0000

This is the fifth part in the my series about building an Android app using Xamarin.Android. I highly recommend reading these first. The previous parts are:

Adding the Add button

Currently we have a nice recycler view showing our dummy counters in card views. So the next step is to allow new counters to be added. The current way to add new items is using a floating action button - a button at the bottom of the screen that when pressed will add the new counter.

First thing to do is add it to the layout in counters_fragment.axml. At the moment this contains a LinearLayout but we’ll need to change that to a FrameLayout so that the add button is correctly placed on top. We’re not using a normal button here, but a FloatingActionButton, a button designed to float above other controls in the view. This means we can add a load of counters and scroll the recycler view up and down without the add button moving from it’s floating location in the bottom right.

<FrameLayout xmlns:android="http://schemas.android.com/apk/res/android"
xmlns:app="http://schemas.android.com/apk/res-auto"
android:layout_width="fill_parent"
android:layout_height="fill_parent">
<android.support.v7.widget.RecyclerView
android:id="@+id/countersRecyclerView"
android:scrollbars="vertical"
android:layout_width="fill_parent"
android:layout_height="fill_parent"
android:background="@color/divider" />
<android.support.design.widget.FloatingActionButton
android:id="@+id/floatingAddNewCounterButton"
android:layout_width="wrap_content"
android:layout_height="wrap_content"
android:layout_gravity="bottom|right"
android:layout_margin="16dp"
android:elevation="4dp"
android:scaleType="fitXY"
android:src="@drawable/ic_add_white_24dp"
app:borderWidth="0dp" />
</FrameLayout>

The icon being used here, ic_add_white_24dp, also comes from Googles material icons at https://design.google.com/icons/. Its a 24dp icon, the recommended size for floating action buttons. We don’t need to set the colour of the action button, this automatically comes from the colours we added from Material Palette in part 1.

Now we have the add button, we need to think about what it needs to do. We need this button to navigate to another activity where the user can enter details about the new counter and save it. MVVMLight has a nice navigation system but we are limited in how we can use it because we are using AppCompat. Luckily I’ve already created a workaround which is documented on another blog post here. To add this working navigation we just need to add the MVVMLight.AppCompat nuget package, and set it up. We start with the set up by changing our BaseActivity to derive from AppCompatActivityBase.

public abstract class BaseActivity : AppCompatActivityBase

Then we need to register the navigation and dialog services with the IoC container. The concrete implementations of the services are platform specific, so we can’t simply register them in the view model locator. Instead we have to expose methods to allow the registration to happen externally so that we can call it from our Android project. The following methods to register need to be added to the ViewModelLoator:

public static void RegisterNavigationService(INavigationService navigationService)
{
SimpleIoc.Default.Register(() => navigationService);
}
public static void RegisterDialogService(IDialogService dialogService)
{
SimpleIoc.Default.Register(() => dialogService);
}

These are then called from our MainActivity in a new constructor:

public MainActivity()
{
var navigationService = new AppCompatNavigationService();
ViewModelLocator.RegisterNavigationService(navigationService);
ViewModelLocator.RegisterDialogService(new AppCompatDialogService());
}

Whilst we’re editing the MainActivity it makes sense to delete the dummy counter code as well - seeing as we are adding the ability to add new counters we don’t need to pre-populate the app with fake counters. Just delete the AddDummyData method and the call to it.

Creating the new counter

To create the new counter we need to define a new activity to allow the user to enter details about the counter. This activity needs a layout, a view model and to be wired into the navigation so that we can navigate to it from our new add button.

Layout

Lets start with the layout. This needs to have text boxes so the user can enter the name and description for the counter, along with a button to create the new counter. We’re not going to bother with a cancel option - the UI will require the create button to be clicked to create the counter with back acting as a cancel, either the back button on the tool bar or the hardware back button.

<?xml version="1.0" encoding="utf-8"?>
<LinearLayout xmlns:android="http://schemas.android.com/apk/res/android"
xmlns:app="http://schemas.android.com/apk/res-auto"
android:orientation="vertical"
android:layout_width="match_parent"
android:layout_height="match_parent">
<android.support.design.widget.AppBarLayout
android:layout_height="wrap_content"
android:layout_width="match_parent"
android:id="@+id/toolbar_layout">
<include
android:id="@+id/toolbar"
layout="@layout/toolbar"
app:layout_scrollFlags="scroll|enterAlways" />
</android.support.design.widget.AppBarLayout>
<LinearLayout
android:orientation="vertical"
android:layout_width="match_parent"
android:layout_height="match_parent"
android:padding="24dp">
<android.support.design.widget.TextInputLayout
android:layout_width="match_parent"
android:layout_height="wrap_content"
android:layout_marginBottom="24dp">
<EditText
android:id="@+id/new_counter_name"
android:layout_width="fill_parent"
android:layout_height="wrap_content"
android:hint="Name" />
</android.support.design.widget.TextInputLayout>
<android.support.design.widget.TextInputLayout
android:layout_width="match_parent"
android:layout_height="wrap_content"
android:layout_marginBottom="24dp">
<EditText
android:id="@+id/new_counter_description"
android:layout_width="fill_parent"
android:layout_height="wrap_content"
android:hint="Description" />
</android.support.design.widget.TextInputLayout>
<Button
android:text="Create Counter"
android:layout_width="match_parent"
android:layout_height="wrap_content"
android:id="@+id/new_counter_create" />
</LinearLayout>
</LinearLayout>

This is a standard app bar layout showing the toolbar, same as the main layout, with text boxes for the name and description, and a ‘Create Counter’ button. The cool thing we’re doing here is the edit boxes. We’re not just using a boring label with a text box below, instead we’re using a TextInputLayout. This wraps the EditText so you see a hint inside the text box, then when you touch inside to enter the text the hint moves to above the box.

View model

Before we create an activity to show this layout, lets create the view model. We start by creating the NewCounterViewModel in our ViewModel folder. The first part of this is to create the properties to hold the name and description fields.

public class NewCounterViewModel : ViewModelBase
{
private string _name;
public string Name
{
get { return _name; }
set { Set(() => Name, ref _name, value); }
}
private string _description;
public string Description
{
get { return _description; }
set { Set(() => Description, ref _description, value); }
}
}

This view model is derived from ViewModelBase, and the property setters use the base class Set method to not only set the value but to raise the relevant property change event.

Now we have the data part sorted, it’s time to add some commands.

public class NewCounterViewModel : ViewModelBase
{
private readonly IDatabaseHelper _databaseHelper;
private readonly IDialogService _dialogService;
private readonly INavigationService _navigationService;
public NewCounterViewModel(IDatabaseHelper databaseHelper, IDialogService dialogService, INavigationService navigationService)
{
_databaseHelper = databaseHelper;
_dialogService = dialogService;
_navigationService = navigationService;
}
private RelayCommand _goBackCommand;
public RelayCommand GoBackCommand => _goBackCommand ?? (_goBackCommand = new RelayCommand(() => _navigationService.GoBack()));
private RelayCommand _addCounterCommand;
public RelayCommand AddCounterCommand => _addCounterCommand ?? (_addCounterCommand = new RelayCommand(async () => await AddCounter()));
private async Task AddCounter()
{
if (string.IsNullOrEmpty(Name))
{
await _dialogService.ShowError("The name must be set", "No name", "OK", null);
return;
}
if (string.IsNullOrEmpty(Description))
{
await _dialogService.ShowError("The description must be set", "No description", "OK", null);
return;
}
await _databaseHelper.AddOrUpdateCounterAsync(new Counter {Name = Name, Description = Description});
_navigationService.GoBack();
}
}

This adds two commands.
The first is the GoBackCommand, which will be wired up to the toolbars back button. This uses the navigation service that is injected into the constructor to navigate back - under the hood this pops the current activity off the stack and returns to the previous one, the same as the hardware back button. The second command, AddCounterCommand will create and add a new command. It starts with some simple validation to ensure the values are set, and if not raises an alert using the MVVMLight dialog service injected into the constructor. If this validation is passed the new counter is created, added to the database using the database helper injected into the constructor, and the activity is popped off the stack using the navigation service.

Like all good coders we should be testing our view model, so lets add NewCounterViewModelTests to the test project.

[TestFixture]
public class NewCounterViewModelTests
{
private Mock<IDatabaseHelper> _mockDatabaseHelper;
private Mock<IDialogService> _mockDialogService;
private Mock<INavigationService> _mockNavigationService;
[SetUp]
public void SetUp()
{
_mockDatabaseHelper = new Mock<IDatabaseHelper>();
_mockDialogService = new Mock<IDialogService>();
_mockNavigationService = new Mock<INavigationService>();
}
[Test]
public void SettingTheNameRaisesAPropertyChangedEvent()
{
var vm = new NewCounterViewModel(_mockDatabaseHelper.Object, _mockDialogService.Object, _mockNavigationService.Object);
vm.MonitorEvents();
vm.Name = "Foo";
vm.ShouldRaisePropertyChangeFor(v => v.Name);
}
[Test]
public void SettingTheDescriptionRaisesAPropertyChangedEvent()
{
var vm = new NewCounterViewModel(_mockDatabaseHelper.Object, _mockDialogService.Object, _mockNavigationService.Object);
vm.MonitorEvents();
vm.Description = "Foo";
vm.ShouldRaisePropertyChangeFor(v => v.Description);
}
[Test]
public void GoBackCommandNavigatesBackwards()
{
var vm = new NewCounterViewModel(_mockDatabaseHelper.Object, _mockDialogService.Object, _mockNavigationService.Object);
vm.GoBackCommand.Execute(null);
_mockNavigationService.Verify(n => n.GoBack(), Times.Once);
}
[Test]
public void AddCommandRaisesAnErrorIfTheNameIsNotSet()
{
var vm = new NewCounterViewModel(_mockDatabaseHelper.Object, _mockDialogService.Object, _mockNavigationService.Object);
vm.Description = "Bar";
vm.AddCounterCommand.Execute(null);
_mockDialogService.Verify(d => d.ShowError(It.IsAny<string>(), It.IsAny<string>(), It.IsAny<string>(), null), Times.Once);
_mockDatabaseHelper.Verify(d => d.AddOrUpdateCounterAsync(It.IsAny<Counter>()), Times.Never);
}
[Test]
public void AddCommandRaisesAnErrorIfTheDescriptionIsNotSet()
{
var vm = new NewCounterViewModel(_mockDatabaseHelper.Object, _mockDialogService.Object, _mockNavigationService.Object);
vm.Name = "Foo";
vm.AddCounterCommand.Execute(null);
_mockDialogService.Verify(d => d.ShowError(It.IsAny<string>(), It.IsAny<string>(), It.IsAny<string>(), null), Times.Once);
_mockDatabaseHelper.Verify(d => d.AddOrUpdateCounterAsync(It.IsAny<Counter>()), Times.Never);
}
[Test]
public void AddComandAddsTheCounterAndNavigatesBack()
{
var vm = new NewCounterViewModel(_mockDatabaseHelper.Object, _mockDialogService.Object, _mockNavigationService.Object);
vm.Name = "Foo";
vm.Description = "Bar";
vm.AddCounterCommand.Execute(null);
_mockDialogService.Verify(d => d.ShowError(It.IsAny<string>(), It.IsAny<string>(), It.IsAny<string>(), null), Times.Never);
_mockDatabaseHelper.Verify(d => d.AddOrUpdateCounterAsync(It.IsAny<Counter>()), Times.Once);
_mockNavigationService.Verify(n => n.GoBack(), Times.Once);
}
}

Because we’re using dependency injection we can mock all the interfaces that are injected into the view model. This means we can test the behaviour of the commands ensuring the GoBackCommand calls the relevant GoBack method on the navigation service, that the navigation doesn’t happen if we add a counter with data missing and that if all the information is there the counter gets added to the database. We can also check that setting the properties raises the relevant property change events. We can get pretty awesome coverage here to ensure out view model works.

The last thing to do with the view model is to add it to our locator. We need this so that the IoC container can resolve the constructor injection.

static ViewModelLocator()
{
...
SimpleIoc.Default.Register<NewCounterViewModel>();
}
public const string NewCounterPageKey = "NewCounterPage";
public static NewCounterViewModel NewCounter => ServiceLocator.Current.GetInstance<NewCounterViewModel>();

Here we register it with the IoC container in the constructor. Unfortunately SimpleIoC only supports singletons, so we have to always deal with a single instance - something we will have to consider later on. Once registered we can expose a static property to return the instance, and a constant that defines a key for it. This key will be registered with the navigation service once we have defined the activity.

Activity

We have our layout and we have our view model, so now we can create the Activity that brings it all together.

Lets start with the basics:

[Activity(Label = "New Counter")]
public class NewCounterActivity : BaseActivity
{
protected override int LayoutResource => Resource.Layout.new_counter;
public override bool OnOptionsItemSelected(IMenuItem item)
{
if (item.ItemId == Android.Resource.Id.Home)
{
ViewModel.GoBackCommand.Execute(null);
return true;
}
return base.OnOptionsItemSelected(item);
}
}

Here we’re creating the activity, derived from our BaseActivity. We override the LayoutResource to point to our new layout. We also override ‘OnOptionsItemSelected’ to detect if the Home button is pressed, executing the command on our view model to navigate backwards.

Next we need to wire up the NewCounterViewModel. We do this in the OnCreate method and store it in a public property (more on this later). As mentioned earlier the IoC container only stores singletons, so we need to clear the data before we can use it to stop the view showing the name and description of the previous counter that was added.

public NewCounterViewModel ViewModel { get; private set; }
protected override void OnCreate(Bundle savedInstanceState)
{
base.OnCreate(savedInstanceState);
ViewModel = ViewModelLocator.NewCounter;
ViewModel.Name = string.Empty;
ViewModel.Description = string.Empty;
}

Then we need to add public properties for the controls on our view.

private EditText _name;
public EditText Name => _name ?? (_name = FindViewById<EditText>(Resource.Id.new_counter_name));
private EditText _description;
public EditText Description => _description ?? (_description = FindViewById<EditText>(Resource.Id.new_counter_description));
private Button _createCounter;
public Button CreateCounter => _createCounter ?? (_createCounter = FindViewById<Button>(Resource.Id.new_counter_create));

These properties will resolve the widgets by looking for them in the layout, and once found stored in a field so we only need to do one lookup.

Lastly we need to bind the controls to the properties on the view model using the MVVMLight binding mechanism. And this is where the public visibility of the properties comes into play.

private readonly List<Binding> _bindings = new List<Binding>();
protected override void OnCreate(Bundle savedInstanceState)
{
...
Bind();
}
private void Bind()
{
_bindings.Add(this.SetBinding(() => ViewModel.Name, () => Name.Text, BindingMode.TwoWay));
_bindings.Add(this.SetBinding(() => ViewModel.Description, () => Description.Text, BindingMode.TwoWay));
CreateCounter.SetCommand(nameof(Button.Click), ViewModel.AddCounterCommand);
}

In out OnCreate method we call a new method, Bind, that uses MVVMLight to bind up the properties. MVVMLight has extension methods for binding properties and commands. We call SetBinding passing in an expression that points to the source for the binding, an expression that points to the target, and the binding mode.

Behind the scenes the SetBinding method will resolve the expressions to the public properties on the object that the extension method is called on. So for example in the first binding it looks at the source expression and finds the ViewModel property (remember from before that we made this property public), and on that view model finds the Name property. It then looks at the target expression and resolves the public Name property on the Activity (the EditText public property we added earlier). It will start by copying the value from the source by evaluating the ViewModel.Name property and setting it on the Name.Text property. Then based on the binding mode it will wire up listeners for value changes. We’re using BindingMode.TwoWay so we need a two-way binding - changes in the source update the target and changes in the target update the source. This means the binding will listen for property change events on the ViewModel and if one is raised for the Name property it will update the control, and it will listen for changes to the text of the Name edit text control, and if the test changes it will update the view model. The SetBinding creates a weak binding, so we need to keep a reference to it to stop the garbage collector from cleaning up. This is what the _bindings list is for.

As well as binding the properties, we need to bind the command. This is done using the SetCommand extension method. This takes the name of the event and a command to bind to. It will find the event with the given name on the object the extension method is called on, and when that event is fired it will execute the command. It also checks the CanExecute status of the command and will enable or disable the button depending on the value. This is a nice way to enable or disable UI functionality from the view model.

Almost there - we now have an activity with a UI and a view model. Now we need to navigate to it.

Navigating to the new activity

Navigation in MVVMLight is based around the idea of registering an activity against a key in your platform specific code, and navigating to that key in the portable code. Registration happens when we first register the navigation service in the MainActivity constructor. We call the Configure method passing in the key we defined in our ViewModelLocator earlier, and the type of Activity we want created when we navigate to this key.

public MainActivity()
{
....
var navigationService = new AppCompatNavigationService();
navigationService.Configure(ViewModelLocator.NewCounterPageKey, typeof(NewCounterActivity));
...
}

We can then add a command to the CountersViewModel to do this navigation using an injected INavigationService.

public CountersViewModel(IDatabaseHelper databaseHelper, INavigationService navigationService)
{
...
_navigationService = navigationService;
...
}
private RelayCommand _addNewCounterCommand;
public RelayCommand AddNewCounterCommand => _addNewCounterCommand ?? (_addNewCounterCommand = new RelayCommand(AddNewCounter));
private void AddNewCounter()
{
_navigationService.NavigateTo(ViewModelLocator.NewCounterPageKey);
}

This command when executed will call the NavigateTo method on the INavigationService passing the key that we defined earlier. Once we make this call the navigation service does everything for us - creating the activity and pushing it onto the top of the navigation stack. And because we’re good developers we can add a unit test to our CountersViewModelTests to verify this as well.

[Test]
public void ExecutingAddNewCounterCommandShouldNavigateToTheNewCounterActivity()
{
var vm = new CountersViewModel(_mockDatabaseHelper.Object, _mockNavigationService.Object);
vm.AddNewCounterCommand.Execute(null);
_mockNavigationService.Verify(n => n.NavigateTo(ViewModelLocator.NewCounterPageKey), Times.Once);
}

Finally we need to bind this command to the UI inside the CountersFragment using the MVVMLight SetCommand extension method we discussed above.

private FloatingActionButton _floatingActionButton;
public override View OnCreateView(LayoutInflater inflater, ViewGroup container, Bundle savedInstanceState)
{
...
_floatingActionButton = view.FindViewById<FloatingActionButton>(Resource.Id.floatingAddNewCounterButton);
_floatingActionButton.SetCommand(nameof(FloatingActionButton.Click), ViewModelLocator.Counters.AddNewCounterCommand);
...
}

Now this is all done we can run the app and test the navigation.

Updating the counters in the UI

If you run and test this you will be able to add a new counter, but the list of counters won’t be updated. Recycler views are not able to automatically detect changes to the underlying collection. The final piece of the adding counters puzzle is to wire this up. The best way to handle this is from the database layer up - we can raise an event in our database helper when the counters are changed, detect this in the counters view model and update our UI.

Let’s start with the database helper by adding an event to our IDatabaseHelper interface:

event EventHandler CountersChanged;

Then we implement it in DatabaseHelper.

public async Task AddOrUpdateCounterAsync(Counter counter)
{
...
OnCountersChanged();
}
public event EventHandler CountersChanged;
private void OnCountersChanged()
{
CountersChanged?.Invoke(this, new EventArgs());
}

Notice the new C# 6 null-conditional operator. This will check the value of CountersChanged and if this is null do nothing, otherwise it will call the Invoke method. Lovely clean code!

We have our event, so lets wire it up in our CountersViewModel to update the counters we expose in the view model.

public CountersViewModel(IDatabaseHelper databaseHelper, INavigationService navigationService)
{
...
_databaseHelper.CountersChanged += async (s, e) => await LoadCountersAsync();
...
}
public async Task LoadCountersAsync()
{
_counters.Clear();
...
}

We’re listening on the event and when it is fired reloading the counters from the SQLite database. We’re re-using the LoadCountersAsync method so we have to tweak it to always clear the collection before populating it with the data loaded from the database helper.

We now have our observable collection changing when the database changes, so we can listen on this to update the view. This can be handled in the CountersAdapter - the component in the recycler view that adapts the counters collection to the UI.

public CountersAdapter()
{
((INotifyCollectionChanged)ViewModelLocator.Counters.Counters).CollectionChanged += OnCollectionChanged;
}
private void OnCollectionChanged(object sender, NotifyCollectionChangedEventArgs notifyCollectionChangedEventArgs)
{
NotifyDataSetChanged();
}

In here we handle the collection change method and when fired call NotifyDataSetChanged, a method that tells the recycler view to update the items in the UI. The Counters property on the CountersViewModel is a ReadOnlyObservableCollection which implements INotifyCollectionChanged explicitly, so we have to cast it to get access to the CollectionChanged event.

Note that this is not the most performant way of doing this. Every action on the collection will cause the entire UI to be updated. The reload is a clear then add of items one by one, so for example if there are 5 counters the CollectionChanged event will be raised 6 times and the UI will be fully rebuilt 6 times. The use of NotifyDataSetChanged will also cause an entire UI rebuild - there are other notify methods that handle single item adds, deletes or moves which could be used to improve performance. We could also improve the performance by implementing our own collection and only raising the event once. In this case we don’t have to worry too much - we don’t need to be lightening fast as will only have a few items.

Lets test it all out

Now everything is wired up, lets take the app for a spin.

Update

After feedback from the comments against this post, lets tweak the UI to have a button on the toolbar to create the counter instead of on the bottom of the view.

First we need to remove the button from the bottom of the new_counter.axml layout file - just need to delete the Button element.

Next we need a menu. The way extra items are added to the toolbar is by creating a menu and inflating it into the toolbar. There is a visual studio template for menus, so right click on the ‘Resources/menu’ folder and add a new item using the Android menu template called new_counter_menu.xml.

In this menu we need to add one item for the new counter button.

<?xml version="1.0" encoding="utf-8" ?>
<menu xmlns:android="http://schemas.android.com/apk/res/android"
xmlns:app="http://schemas.android.com/apk/res-auto">
<item android:id="@+id/action_create_counter"
app:showAsAction="always"
android:icon="@drawable/ic_done_white_48dp"/>
</menu>

The icon being used here is the done icon from Google’s material icons mentioned above. Download this one and copy it to the various drawable folders. Notice as well the showAsAction property comes from the xmlns:app="http://schemas.android.com/apk/res-auto" namespace - if you use it from the android namespace then the menu item will appear on the overflow menu without an icon.

Now we have our menu, we need to wire it up in the NewCounterActivity. We can start here by deleting the CreateCounter button and all references to it. Then we need to create the new menu in the toolbar. This is done by overriding the OnCreateMenuItems method and loading our new menu into the toolbar in there.

public override bool OnCreateOptionsMenu(IMenu menu)
{
base.OnCreateOptionsMenu(menu);
Toolbar.InflateMenu(Resource.Menu.new_counter_menu);
return true;
}

Last thing to do is to handle the menu click in the OnOptionsItemSelected method. When the new menu item is selected we need to execute the same command the previous button used.

public override bool OnOptionsItemSelected(IMenuItem item)
{
switch (item.ItemId)
{
case Android.Resource.Id.Home:
ViewModel.GoBackCommand.Execute(null);
return true;
case Resource.Id.action_create_counter:
ViewModel.AddCounterCommand.Execute(null);
return true;
default:
return base.OnOptionsItemSelected(item);
}
}

Done!

The code for this can be found in GitHub on the Part5 branch at https://github.com/jimbobbennett/StupendousCounter/tree/Part5

In the next part we’ll work on deleting counters.

MVVMLight navigation and AppCompatActivity

Mon, 08 Feb 2016 00:16:50 +0000

As much as I’m loving MVVMLight it does have some limitations. The one I’ve hit recently is how well it doesn’t work when you are using AppCompat (though to be honest I think it’s more down to how hacky AppCompat seems to be).

I’m using AppCompatActivity as my base activity class to get an action bar supported on older API versions. The problem is MVVMLight implements it’s own ActivityBase which is derived from Activity, and it relies on a set of static methods and properties on this to handle things like navigation and dialogs. If you want to use AppCompatActivity you are stuck - navigation will not work.

Seems to get #mvvmlight working with AppCompatActivity I have to write my own NavigationService. @LBugnion any thoughts?
— Jim Bennett (@jimbobbennett) February 7, 2016

@jimbobbennett I don't have a better proposal at this point. Android is severely broken when it comes to launching new activities.
— Laurent Bugnion (@LBugnion) February 7, 2016

Luckily MVVMLight is really well designed with good separation of concerns, so everything is done via interface. This means we can write our own implementation of the navigation service and dialog service that will work with AppCompatActivity and use it in our code. Being open source as well we can take the existing source and tweak it slightly to make it work for us. It’s a bit of a hack but it works.

The first thing to do is to define our own base activity class that has the same functionality as the ActivityBase, but derived from AppCompatActivity.

using Android.Support.V7.App;
namespace JimBobBennett.MvvmLight.AppCompat
{
public abstract class AppCompatActivityBase : AppCompatActivity
{
public static AppCompatActivityBase CurrentActivity { get; private set; }
internal string ActivityKey { get; private set; }
internal static string NextPageKey { get; set; }
public static void GoBack()
{
CurrentActivity?.OnBackPressed();
}
protected override void OnResume()
{
CurrentActivity = this;
if (string.IsNullOrEmpty(ActivityKey))
{
ActivityKey = NextPageKey;
NextPageKey = null;
}
base.OnResume();
}
}
}

This is identical to the MVVMLight ActivityBase except the base class is AppCompatActivity.

Now we have our base activity we can use it for the navigation service. This is identical to the MVVMLight Android NavigationService except instead of accessing the statics on AppCompatActivityBase instead of on ActivityBase.

using System;
using System.Collections.Generic;
using Android.Content;
using GalaSoft.MvvmLight.Views;
namespace JimBobBennett.MvvmLight.AppCompat
{
public class AppCompatNavigationService : INavigationService
{
private readonly Dictionary<string, Type> _pagesByKey = new Dictionary<string, Type>();
private readonly Dictionary<string, object> _parametersByKey = new Dictionary<string, object>();
private const string RootPageKey = "-- ROOT --";
private const string ParameterKeyName = "ParameterKey";
public string CurrentPageKey => AppCompatActivityBase.CurrentActivity.ActivityKey ?? RootPageKey;
public void GoBack()
{
AppCompatActivityBase.GoBack();
}
public void NavigateTo(string pageKey)
{
NavigateTo(pageKey, null);
}
public void NavigateTo(string pageKey, object parameter)
{
AppCompatActivityBase.CurrentActivity.RunOnUiThread(() =>
{
if (AppCompatActivityBase.CurrentActivity == null)
throw new InvalidOperationException("No CurrentActivity found");
lock (_pagesByKey)
{
if (!_pagesByKey.ContainsKey(pageKey))
throw new ArgumentException($"No such page: {pageKey}. Did you forget to call NavigationService.Configure?", nameof(pageKey));
var intent = new Intent(AppCompatActivityBase.CurrentActivity, _pagesByKey[pageKey]);
if (parameter != null)
{
lock (_parametersByKey)
{
var guid = Guid.NewGuid().ToString();
_parametersByKey.Add(guid, parameter);
intent.PutExtra(ParameterKeyName, guid);
}
}
AppCompatActivityBase.CurrentActivity.StartActivity(intent);
AppCompatActivityBase.NextPageKey = pageKey;
}
});
}
public void Configure(string key, Type activityType)
{
lock (_pagesByKey)
{
if (_pagesByKey.ContainsKey(key))
_pagesByKey[key] = activityType;
else
_pagesByKey.Add(key, activityType);
}
}
public object GetAndRemoveParameter(Intent intent)
{
if (intent == null)
throw new ArgumentNullException(nameof(intent), "This method must be called with a valid Activity intent");
var stringExtra = intent.GetStringExtra(ParameterKeyName);
if (string.IsNullOrEmpty(stringExtra))
return null;
lock (_parametersByKey)
return _parametersByKey.ContainsKey(stringExtra) ? _parametersByKey[stringExtra] : null;
}
public T GetAndRemoveParameter<T>(Intent intent)
{
return (T)GetAndRemoveParameter(intent);
}
}
}

We then do the same for the dialog service:

using System;
using System.Threading.Tasks;
using Android.App;
using Android.Content;
using GalaSoft.MvvmLight.Views;
namespace JimBobBennett.MvvmLight.AppCompat
{
public class AppCompatDialogService : IDialogService
{
public Task ShowError(string message, string title, string buttonText, Action afterHideCallback)
{
var afterHideCallbackWithResponse = (Action<bool>) (r =>
{
if (afterHideCallback == null)
return;
afterHideCallback();
afterHideCallback = null;
});
var dialog = CreateDialog(message, title, buttonText, null, afterHideCallbackWithResponse);
dialog.Dialog.Show();
return dialog.Tcs.Task;
}
public Task ShowError(Exception error, string title, string buttonText, Action afterHideCallback)
{
var afterHideCallbackWithResponse = (Action<bool>)(r =>
{
if (afterHideCallback == null)
return;
afterHideCallback();
afterHideCallback = null;
});
var dialog = CreateDialog(error.Message, title, buttonText, null, afterHideCallbackWithResponse);
dialog.Dialog.Show();
return dialog.Tcs.Task;
}
public Task ShowMessage(string message, string title)
{
var dialog = CreateDialog(message, title);
dialog.Dialog.Show();
return dialog.Tcs.Task;
}
public Task ShowMessage(string message, string title, string buttonText, Action afterHideCallback)
{
var afterHideCallbackWithResponse = (Action<bool>)(r =>
{
if (afterHideCallback == null)
return;
afterHideCallback();
afterHideCallback = null;
});
var dialog = CreateDialog(message, title, buttonText, null, afterHideCallbackWithResponse);
dialog.Dialog.Show();
return dialog.Tcs.Task;
}
public Task<bool> ShowMessage(string message, string title, string buttonConfirmText, string buttonCancelText, Action<bool> afterHideCallback)
{
var afterHideCallbackWithResponse = (Action<bool>)(r =>
{
if (afterHideCallback == null)
return;
afterHideCallback(r);
afterHideCallback = null;
});
var dialog = CreateDialog(message, title, buttonConfirmText, buttonCancelText ?? "Cancel", afterHideCallbackWithResponse);
dialog.Dialog.Show();
return dialog.Tcs.Task;
}
public Task ShowMessageBox(string message, string title)
{
return ShowMessage(message, title);
}
private static AlertDialogInfo CreateDialog(string content, string title, string okText = null, string cancelText = null, Action<bool> afterHideCallbackWithResponse = null)
{
var tcs = new TaskCompletionSource<bool>();
var builder = new AlertDialog.Builder(AppCompatActivityBase.CurrentActivity);
builder.SetMessage(content);
builder.SetTitle(title);
var dialog = (AlertDialog)null;
builder.SetPositiveButton(okText ?? "OK", (d, index) =>
{
tcs.TrySetResult(true);
if (dialog != null)
{
dialog.Dismiss();
dialog.Dispose();
}
if (afterHideCallbackWithResponse == null)
return;
afterHideCallbackWithResponse(true);
});
if (cancelText != null)
{
builder.SetNegativeButton(cancelText, (d, index) =>
{
tcs.TrySetResult(false);
if (dialog != null)
{
dialog.Dismiss();
dialog.Dispose();
}
if (afterHideCallbackWithResponse == null)
return;
afterHideCallbackWithResponse(false);
});
}
builder.SetOnDismissListener(new OnDismissListener(() =>
{
tcs.TrySetResult(false);
if (afterHideCallbackWithResponse == null)
return;
afterHideCallbackWithResponse(false);
}));
dialog = builder.Create();
return new AlertDialogInfo
{
Dialog = dialog,
Tcs = tcs
};
}
private struct AlertDialogInfo
{
public AlertDialog Dialog;
public TaskCompletionSource<bool> Tcs;
}
private sealed class OnDismissListener : Java.Lang.Object, IDialogInterfaceOnDismissListener
{
private readonly Action _action;
public OnDismissListener(Action action)
{
_action = action;
}
public void OnDismiss(IDialogInterface dialog)
{
_action();
}
}
}
}

To use these all we need to do is derive all our activities from AppCompatActivityBase and register the AppCompatNavigationService and AppCompatDialogService in our IoC container as implementations of INavigationService and IDialogService.

The code for this is up on GitHub here: https://github.com/jimbobbennett/JimBobBennett.MvvmLight.AppCompat

I’ve also created a NuGet package available here: https://www.nuget.org/packages/JimBobBennett.MvvmLight.AppCompat/ or searchable from the Package Manager as JimBobBennett.MvvmLight.AppCompat.

Update

Thanks to Samuel Debruyn for adding an update to this package.

MVVMLight has a helper class to aid in running code on the UI thread called DispatcherHelper. This will verify the current activity and use that to run the specified action on the UI thread using the RunOnUIThread method.

Unfortunately for us this relies on the CurrentActivity property on ActivityBase, something we are not using. Sam has added an AppCompatDispatcherHelper which mimics the logic of the DispatcherHelper but using the AppCompatActivityBase.CurrentActivity instead.

The code on GitHub and NuGet package have both been updated with these changes.

Building a Xamarin Android app - part 4

Mon, 01 Feb 2016 07:36:22 +0000

This is the fourth part in the my series about building an Android app using Xamarin.Android. You can find the first part here, the second part here and the third part here, and I highly recommend reading these first.

Binding our view models to the UI

We have our models, we have our view models, now to work on the views! First thing we need to do is a bit of a tidy up - the default UI code we’ve picked up from our templates doesn’t match what we want to show in screen, so lets start by clearing everything up a bit.

The UI we have has a navigation drawer with 2 screens you can select, as well as some sub menu options. The screen selection options load one of two possible fragments into our UI. We can repurpose these - one fragment to show our counters and one to show an about screen so we can tell the world who created such a stupendous counter app!

Stating with the first fragment we can rename the class from Fragment1 to CountersFragment, as well as renaming the associated layout from fragment1.axml to counters_fragment.axml. After renaming the layout file we also need to change the id that is used in the CountersFragment.OnCreateView method to reflect the new name:

return inflater.Inflate(Resource.Layout.counters_fragment, null);

For the second one we can rename it from Fragment2 to AboutFragment and fragment2.axml to about_fragment.axml, and again updating the id:

return inflater.Inflate(Resource.Layout.about_fragment, null);

Now is also a good time to update the menu in menu\nav_menu.xml to remove the unwanted sub items and rename the main items:

<?xml version="1.0" encoding="UTF-8" ?>
<menu xmlns:android="http://schemas.android.com/apk/res/android">
<group android:checkableBehavior="single">
<item
android:id="@+id/nav_counters"
android:icon="@drawable/ic_add_circle_black_48dp"
android:title="Counters" />
<item
android:id="@+id/nav_about"
android:icon="@drawable/ic_settings_black_48dp"
android:title="About" />
</group>
</menu>

You’ll notice the icons have changed to items not in our drawables folder, so we’ll need to add these. The icons we’re using are from the Google material icons - you can download them from https://design.google.com/icons/. You’ll need to download the ‘add circle’ and ‘settings’ icons as pngs and copy them from the drawables folders of the downloads to the same named drawables folders locally, then add them to the project.

Changing the menu ids will break our MainActivity so we need to fix the OnCreate method by changing the subscription to the NavigationView.NavigationItemSelected event. At the same time we can remove the snack bar call as we don’t want one popping up when we change the fragment.

navigationView.NavigationItemSelected += (sender, e) =>
{
e.MenuItem.SetChecked(true);
switch (e.MenuItem.ItemId)
{
case Resource.Id.nav_counters:
ListItemClicked(0);
break;
case Resource.Id.nav_about:
ListItemClicked(1);
break;
}
drawerLayout.CloseDrawers();
};

CountersFragment

In this fragment we want to display a list of all the counters that we have stored. The latest and greates way to do this is with a RecyclerView which is documented here on the Xamarin docs. This is like a list view but enforces good design and ensures the views created are always re-used when they go off screen reducing the memory footprint. It also enforces the use of the view holder pattern to futher improve performance.

To use the recycler view we need to add a nuget package to our StupendoudCounter.Droid project - Xamarin.Android.Support.v7.RecyclerView. This provides the recycler view for all versions of Android from API level 7 and above.

Once we have our nuget package installed we can add the recycler view to our UI and create it’s backing field. In counters_fragment.axml add the recycler view:

<?xml version="1.0" encoding="utf-8"?>
<LinearLayout xmlns:android="http://schemas.android.com/apk/res/android"
android:orientation="vertical"
android:layout_width="fill_parent"
android:layout_height="fill_parent">
<android.support.v7.widget.RecyclerView
android:id="@+id/countersRecyclerView"
android:scrollbars="vertical"
android:layout_width="fill_parent"
android:layout_height="fill_parent" />
</LinearLayout>

In CountersFragment add a field for the recycler view and find it from the view in the OnCreateView method:

using Android.Support.V7.Widget;
...
private RecyclerView _recyclerView;
public override View OnCreateView(LayoutInflater inflater, ViewGroup container, Bundle savedInstanceState)
{
var ignored = base.OnCreateView(inflater, container, savedInstanceState);
var view = inflater.Inflate(Resource.Layout.counters_fragment, null);
_recyclerView = view.FindViewById<RecyclerView>(Resource.Id.countersRecyclerView);
return view;
}

To use this recycler view we need to use or implement a few things:

A layout manager
An adapter
A view holder

Layout manager

Each instance of a recycler view has a layout manager - this determines how the items in the view are laid out. Android provides 3 basic ones which are good enough for most cases, but you can implement your own if you wish. There’s LinearLayoutManager which displays the items in a horizontal or vertical list, GridLayoutManager that displayes the items in a grid and StaggeredGridLayoutManager which displays them in a grid with uneven rows or columns. For what we need the LinearLayoutManager is good enough, so we can create one and set it on our recycler view:

_recyclerView.SetLayoutManager(new LinearLayoutManager(Context, LinearLayoutManager.Vertical, false));

The three parameters for the constructor are the current context which we can get from the Context property, the orientation for which we are using the Vertical constant defined on LinearLayoutManager, and a boolean to say if the items should be reveresed or not when we show them - so should we show the items in our list from top to bottom, or bottom to top (reverse is useful when adding new items to the end of a list but showing them in latest-first order, such as an email client order by date).

Adapter

The adapter’s job is to act like a view model for the recycler view - it needs to know about the collection of items we are showing in the list and be able to tell the recycler view how many there are and needs to be able to create the views for items in the collection where necessary or recycle them to be used by other items in the collection. All adapters need three things - a class derived from RecyclerView.Adapter, a view to create to show the item and a view holder that maps the items in the collection to the view. To create the adapter create a class called CountersAdapter:

using Android.Support.V7.Widget;
using Android.Views;
using StupendousCounter.Core.ViewModel;
namespace StupendousCounter.Droid.Fragments
{
public class CountersAdapter : RecyclerView.Adapter
{
public override void OnBindViewHolder(RecyclerView.ViewHolder holder, int position)
{
var item = ViewModelLocator.Counters.Counters[position];
((CounterViewHolder) holder).BindCounterViewModel(item);
}
public override RecyclerView.ViewHolder OnCreateViewHolder(ViewGroup parent, int viewType)
{
var itemView = LayoutInflater.From(parent.Context).Inflate(Resource.Layout.counter_view, parent, false);
return new CounterViewHolder(itemView);
}
public override int ItemCount => ViewModelLocator.Counters.Counters.Count;
}
}

When we override RecyclerView.Adapter we have to implement three things - ItemCount, OnCreateViewHolder and OnBindVewHolder.

ItemCount just needs to return the number of items in the collection. This just returns the count from the CountersViewModel instance from the static ViewModelLocator.

OnCreateViewHolder is called whenever an item in the recycler view is created for the first time. This needs to create a view and wrap it in a class derived from RecyclerView.ViewHolder.

OnBindViewHolder is responsible for updating the view holder to reflect the relevant item in the collection. The item is given by the position parameter - this indicates the position in the collection of the item we need to show in the view. In our code we are using this to get the item from our view model which we access using the static ViewModelLocator, and this is passed to a method on the view holder to populate it. In a lot of code you will see the view holder updated directly here with the controls in the view holder exposed as public properties, but I prefer to encapsulate the controls inside the view holder and have a single method to call to update the view. This means if the view changes the adapter doesn’t need to change.

These last two are the basis of how the recycler view works - it calls OnCreateViewHolder to create just enough views to fill the screen, then calls OnBingViewHolder to show the data. As the collection is scrolled instead of creating new views, the views that are no longer visible are re-used. So if you scroll down a a view disappears off the top it is moved to the bottom to remove the overhead of creating a new view. To make sure it shows the right data OnBindViewHolder is called to update the view to show the correct data.

To create the view add a new layout called counter_view.xml:

<?xml version="1.0" encoding="utf-8"?>
<android.support.v7.widget.CardView xmlns:android="http://schemas.android.com/apk/res/android"
xmlns:card_view="http://schemas.android.com/apk/res-auto"
android:layout_width="match_parent"
android:layout_height="wrap_content"
card_view:cardElevation="8dp"
card_view:cardCornerRadius="8dp"
android:layout_marginLeft="8dp"
android:layout_marginRight="8dp"
android:layout_marginTop="8dp">
<GridLayout
android:minWidth="25px"
android:minHeight="25px"
android:layout_width="match_parent"
android:layout_height="wrap_content"
android:columnCount="3"
android:rowCount="1">
<TextView
android:id="@+id/counter_value"
android:layout_width="wrap_content"
android:layout_height="wrap_content"
android:text="10"
android:layout_row="0"
android:layout_column="1"
android:textSize="48sp"
android:layout_gravity="center_vertical"
android:layout_marginRight="16sp"
android:textColor="@color/primaryDark" />
<ImageButton
android:id="@+id/counter_increment"
android:layout_width="wrap_content"
android:layout_height="wrap_content"
android:layout_row="0"
android:layout_column="2"
android:layout_gravity="center_vertical"
android:layout_margin="16sp"
android:src="@drawable/ic_add_circle_black_48dp"
android:background="#00000000"/>
<LinearLayout
android:orientation="vertical"
android:layout_gravity="fill"
android:layout_row="0"
android:layout_column="0"
android:padding="16sp">
<TextView
android:id="@+id/counter_name"
android:layout_width="wrap_content"
android:layout_height="wrap_content"
android:text="Name"
android:padding="4sp"
android:textSize="24sp"
android:textColor="@color/primaryText" />
<TextView
android:id="@+id/counter_description"
android:layout_width="wrap_content"
android:layout_height="wrap_content"
android:text="The counters description"
android:padding="4sp"
android:textSize="16sp" />
</LinearLayout>
</GridLayout>
</android.support.v7.widget.CardView>

This view uses a CardView which is documented here on the Xamarin docs. Inside the CardView there is a GridLayout and LinearLayout to layout the various widgets, three TextViews to show the counter details and value, and an ImageButton to allow the counter to be incremented. The view looks like this:

The GridLayout is used to layout 3 columns - one for the counter details, one for the count and one for the increment button. Inside the first column is the LinearLayout that shows the counters name and description as a vertical layout.

ViewHolder

The view holder’s job is to create backing fields for the controls in the view to improve performance by only having to find the controls by id once per instance of the view. To create the view holder create a class called CounterViewHolder:

using System;
using System.ComponentModel;
using Android.App;
using Android.Graphics;
using Android.Support.V4.Content;
using Android.Support.V7.Widget;
using Android.Views;
using Android.Widget;
using StupendousCounter.Core.ViewModel;
namespace StupendousCounter.Droid.Fragments
{
public class CounterViewHolder : RecyclerView.ViewHolder
{
private readonly TextView _name;
private readonly TextView _description;
private readonly TextView _value;
private CounterViewModel _counterViewModel;
public CounterViewHolder(View itemView) : base(itemView)
{
_name = itemView.FindViewById<TextView>(Resource.Id.counter_name);
_description = itemView.FindViewById<TextView>(Resource.Id.counter_description);
_value = itemView.FindViewById<TextView>(Resource.Id.counter_value);
var increment = itemView.FindViewById<ImageButton>(Resource.Id.counter_increment);
increment.SetColorFilter(new Color(ContextCompat.GetColor(Application.Context, Resource.Color.primaryDark)));
increment.Click += IncrementOnClick;
}
private void IncrementOnClick(object sender, EventArgs eventArgs)
{
_counterViewModel.IncrementCommand.Execute(null);
}
public void BindCounterViewModel(CounterViewModel counterViewModel)
{
if (_counterViewModel != null)
_counterViewModel.PropertyChanged -= CounterViewModelOnPropertyChanged;
_counterViewModel = counterViewModel;
_counterViewModel.PropertyChanged += CounterViewModelOnPropertyChanged;
_name.Text = counterViewModel.Name;
_description.Text = counterViewModel.Description;
_value.Text = counterViewModel.Value;
}
private void CounterViewModelOnPropertyChanged(object sender, PropertyChangedEventArgs args)
{
if (args.PropertyName == nameof(CounterViewModel.Value))
_value.Text = _counterViewModel.Value;
}
}
}

This class derives from RecyclerView.ViewHolder. In the constructor a view is passed in - and this is the view created by our adapter. In here we are manually going to wire up the values for the counter view model to the view. Although we are using MVVMLight for our view models which exposes a binding mechanism this currently doesn’t work with view holders (although Laurent tells me it should do in a couple of weeks time so stay tuned for an update), so we have to do it all manually.

The first thing we do is grab the name, description and value text edit fields and store these. FindById is slow, hence why we only want to do this once per view instance and store the found controls. For the image button we don’t need to store it in our class, we just need to wire up the Click event so we can respond to it. We also call SetColorFilter on the button - this is because the icon that was downloaded from Google material design images is a black button and we want it to match our theme. SetColorFilter will change the colour of the button to the given colour, giving a nice purple button.

In the adapter in the OnBindViewHolder method we delegated the updating of the UI to a method on the view holder, and this is implemented here in the BindCounterViewModel method. This method takes a CounterViewModel that refers to the item in the relevant position in the collection, and this is stored in a field. The name, description and value controls are updated to match the view model. We also subscribe to the PropertyChanged event so that the value can be updated when it changes on the view model - such as when the increment button is pressed. To avoid the wrong counters being incremented we also unsubscribe from this event from the view model stored in our field if it is set before we update it to store the one passed in.

Once the view model is stored, we can increment it when the increment button is clicked. In the click event handler (IncrementOnClick) we execute the IncrementCommand which will cause the value to increment and be updated in SQLite as shown in the previous post. This will also cause the PropertyChange event to be fired for the Value property, which we handle and update the UI to reflect the new value.

Lets try it out

That should be everything we need to do to show some dummy data and increment the counters, so lets build it, run it and try it out. Click on the plus button to increment each counter, then try closing and re-opening the app - you’ll notice the values are persisted thanks to our SQLite db.

The code for this can be found in GitHub on the Part4 branch at https://github.com/jimbobbennett/StupendousCounter/tree/Part4

In the next part we’ll work on adding an Add button to add a new counter.

Continuing on from the last post where I mentioned what I was listening to whilst developing, today I'm listening to Sleeping at Last

Note - these are an affiliate links - if you click them and buy I get a small cut.

I'm now a Xamarin MVP!

Mon, 01 Feb 2016 07:06:17 +0000

I’m very proud to have been recognised as a Xamarin MVP! Thanks to whoever nominated me. I hope the work I’ve been doing for the community through my talks and blog has helped some people, and will continue to do so in the future. Look out for my up coming talks in New Zealand and at Evolve, and my upcoming blog posts and (hopefully) book.

Building a Xamarin Android app - part 3

Sat, 30 Jan 2016 17:51:36 +0000

This is the third part in the my series about building an Android app using Xamarin.Android. You can find the first part here and the second part here, and I highly recommend reading these first.

Data binding

In the first 2 parts of this we created our basic app shell and built a core library with models that used SQLite to store and retrieve our data. We’re now going to expand on this by adding the layer in-between to bind the data to our UI.

Having been a WPF developer in the past I’m a big fan of the MVVM pattern, so I’m planning on using the same pattern here. I won’t go into the details of this pattern, so if you’ve never come across it before it’s worth reading up as it’s a useful pattern supported natively in Xamarin.Forms and on iOS and Android using tools like MVVMLight or MVVMCross. To help with this I’m going to use MVVMLight by Laurent Bugnion. As the name suggests this is a light-weight MVVM library which provides the basic toolbox to allow you to use the pattern without constraining you to a particular architecture. MVVMLight has two different nuget packages to install - one for the Android app and one for our core project.

For StupendousCounter.Droid install MVVMLight. This has all the platform specific code in (so for an Android project you get a library with Android specific code in) that allows for things like UI binding - this needs to know about the controls so needs to be platform-specific. When installing this you may get an error about not being able to find App.Xaml - just ignore this as it isn’t relevant for Android projects. Once this is installed you will see a new folder created called ViewModel that contains a dummy view model and an example ViewModelLocator. We’ll come back to these later on.

For StupendousCounter.Core install the libs only one - MVVMLightLibs. This has things like the base view model but nothing platform specific.

Showing some data

Lets start by showing some data on screen. To make it easier we’ll just mock some up on startup if we don’t have an existing counters. In our MainActivity.cs we can add some dummy data if we are in debug (to save accidentally leaving this in in release code) just after we’ve setup the DatabaseHelper:

private static async Task AddDummyData()
{
var dbHelper = new DatabaseHelper();
if (!(await dbHelper.GetAllCountersAsync()).Any())
{
var counter1 = new Counter
{
Name = "Monkey Count",
Description = "The number of monkeys",
Value = 10
};
var counter2 = new Counter
{
Name = "Playtpus Count",
Description = "The number of duck-billed platypuses",
Value = 4
};
await dbHelper.AddOrUpdateCounterAsync(counter1);
await dbHelper.AddOrUpdateCounterAsync(counter2);
}
}

And call this if we’re in debug in our ‘OnCreate’ method:

protected override async void OnCreate(Bundle savedInstanceState)
{
base.OnCreate(savedInstanceState);
var path = System.Environment.GetFolderPath(System.Environment.SpecialFolder.Personal);
var dbPath = Path.Combine(path, "counters.db3");
DatabaseHelper.CreateDatabase(dbPath);
# if DEBUG
await AddDummyData();
# endif

Notice how we’re awaiting the call to AddDummyData? Yup - this is an async method we’re in. In the default OnCreate it’s not async - but we can make it so by just adding the async keyword and it just works. Which is really nice.

We have some data, so lets create our first view model to display it. When we added MVVMLight to the StupendousCounter.Droid project it created a dummy MainViewModel and ‘ViewModelLocator’. These are useful starting points for our code but they are not in the right place - one of the big upsides of MVVM is the separation of concerns that is gives us, so the UI is completely independent of the Model (which contains our data) and the view model (which is a converter that converts from the UI view of our data to the Models view). Despite the view model converting from model to UI it should never contain any UI specific code - it converts the Models into a set of properties and commands that are bound to the UI in the View and allows the UI to get or set data or perform actions. Properties are simple data types that the view can show, update and detect updates to based off the INotifyPropertyChanged interface. Commands are implementations of ICommand that allow an event raised on the UI to run an action including checking if that action is allowed based on some criteria. These are all non-UI specific pieces and therefore do not need to be in the UI layer, nor do they need to be platform specific. So it’s better to put them in the Core project so that if we want to port this app to iOS we don’t have to re-implement the view models, just the Views. We can also then write unit tests against them simulating the View’s interaction by getting or setting properties, detecting the PropertyChanged event on the INotifyPropertyChanged interface and executing commands.

To do this just drag the folder to the StupendousCounter.Core project and delete it from StupendousCounter.Droid. Then just update the namespaces in the 2 files to match the new location - using Resharper of course!

We need a couple of view models - one for the individual counters and one for the overall collection of counters.

CounterViewModel

Lets start with the CounterViewModel, a view model that wraps a Counter:

using System.Threading.Tasks;
using GalaSoft.MvvmLight;
using GalaSoft.MvvmLight.Command;
namespace StupendousCounter.Core.ViewModel
{
public class CounterViewModel : ViewModelBase
{
private readonly Counter _counter;
private readonly IDatabaseHelper _databaseHelper;
public CounterViewModel(Counter counter, IDatabaseHelper databaseHelper)
{
_counter = counter;
_databaseHelper = databaseHelper;
}
public string Name => _counter.Name;
public string Description => _counter.Description;
public string Value => _counter.Value.ToString("N0");
private RelayCommand _incrementCommand;
public RelayCommand IncrementCommand => _incrementCommand ?? (_incrementCommand = new RelayCommand(async () => await IncrementAsync()));
private async Task IncrementAsync()
{
await _databaseHelper.IncrementCounterAsync(_counter);
RaisePropertyChanged(() => Value);
}
}
}

This view model derives from GalaSoft.MvvmLightViewModelBase. This base class provides a few useful bits of functionality including an implementation of INotifyPropertyChanged to make it easy to raise events when properties change. It has other cool stuff as well, but that’s outside the scope of this but it’s worth reading up on it.

The constructor takes a Counter that will provide all our data, and an IDatabaseHelper that we can use to update the counters value. Note that its the interface, not the concrete DatabaseHelper that we use so that we can mock it later for unit tests.

The Name and Description properties are simple read-only pass throughs to the properties on the underlying Counter. The Value property is slightly more complicated - I’m converting it to a string with no decimal places so it can be shown in a text view. I’m using the new, cleaner C# 6 expression bodied property syntax for these which is so much cleaner than writing the properties out in full. These are read only as there is no need to update them directly - the only property that we can update is the value when the counter is incremented and we don’t want to do this with a direct property update, instead we want to do this in the IncrementCommand. This command uses a MVVMLight RelayCommand to wrap an async call to our Increment method. This will increment the counter in the SQLite db using our database helper, then call RaisePropertyChanged to indicate back to the view that the value has changed so the UI should update what it shows on screen. One thing I still think is odd is that there isn’t an implementation of ICommand in the .Net framework - you either have to write your own or use one from a library like MVVMLight. Seems like a serious omission to me.

Unit testing the CounterViewModel

We’ve now got the view model for our counter, so before we use it we should test it (I know some people will say we should have written our tests before writing the view model but hey ho). For this we need a basic class library called StupendousCounter.Core.Tests.

We can add the same NUnit and FluentAssertions nuget packages as we did for the Android bases tests, and MVVMLightLibs. We also want to install Moq, an enjoyable mocking library that we can use to mock out the database layer. The CommonServiceLocator shown in the picture below is a dependency of MVVMLightLibs and is installed automatically.

Now we have our packages installed, lets delete the auto created Class1.cs and add our CounterViewModelTests class. I like to keep the structure of the tests project the same as the project it’s testing so I’ve created it in the ViewModel folder.

using FluentAssertions;
using Moq;
using NUnit.Framework;
using StupendousCounter.Core.ViewModel;
namespace StupendousCounter.Core.Tests.ViewModel
{
[TestFixture]
public class CounterViewModelTests
{
private Mock<IDatabaseHelper> _mockDatabaseHelper;
private CounterViewModel CreateCounterViewModel(int value = 10, string description = "Bar", string name = "Foo")
{
var counter = new Counter { Name = name, Description = description, Value = value };
return new CounterViewModel(counter, _mockDatabaseHelper.Object);
}
[SetUp]
public void SetUp()
{
_mockDatabaseHelper = new Mock<IDatabaseHelper>();
}
[Test]
public void NameShouldComeFromTheCounter()
{
CreateCounterViewModel().Name.Should().Be("Foo");
}
[Test]
public void DescriptionShouldComeFromTheCounter()
{
CreateCounterViewModel().Description.Should().Be("Bar");
}
[Test]
public void ValueShouldComeFromTheCounter()
{
CreateCounterViewModel().Value.Should().Be("10");
}
[Test]
public void ExecutingTheIncrementCommandShouldIncrementTheValueInTheDatabaseHelper()
{
_mockDatabaseHelper.Setup(d => d.IncrementCounterAsync(It.IsAny<Counter>())).Callback<Counter>(c => ++c.Value);
var counter = new Counter {Value = 10};
var vm = new CounterViewModel(counter, _mockDatabaseHelper.Object);
vm.IncrementCommand.Execute(null);
_mockDatabaseHelper.Verify(d => d.IncrementCounterAsync(counter));
vm.Value.Should().Be("11");
}
[Test]
public void ExecutingTheIncrementCommandShouldRaiseAPropertyChangeForValue()
{
var vm = CreateCounterViewModel();
vm.MonitorEvents();
vm.IncrementCommand.Execute(null);
vm.ShouldRaisePropertyChangeFor(v => vm.Value);
}
}
}

Some of the tests should be self-explanatory. The xxxShouldComeFromTheCounter tests verify that the values on the view model actually come from the counter. These are great sanity tests as it’s easy to copy/past properties when creating lots of passthroughs in bigger view models and forget to update the underlying property name. For the value test we are expecting a string value, so we’re comparing against a string representation of the set value.

ExecutingTheIncrementCommandShouldIncrementTheValueInTheDatabaseHelper is a bit more interesting. This tests 2 things - firstly that the database helper is used to increment the counter (remember our architecture is such that the incrementation is done in the database helper to ensure we store the history, and secondly that the value is updated in the view model. It’s in this test the Moq is used. In the SetUp method a Mock<IDatabaseHelper> is created - this is a dummy object that we can use to provide our own implementation of the interface. It exposes an Object property that is the interface we’re implementing and allows us to setup methods and properties to do what we want, and verify calls against them. In this case in the test were setting up the IncrementCounterAsync method to increment the counters value. When we execute the IncrementCommand it calls this method on the database helper stored in the view model, which is our mock, so our method gets called. After the command is executed we verify that the method on the Mock was actually called, and validate that the value was incremented. Mocks are really handy as we’ve managed to test code that calls the database helper without actually needing to set up a SQLite db.

The final test ExecutingTheIncrementCommandShouldRaiseAPropertyChangeForValue is interesting as well. This is to validate that our UI is going to be updated. We can take it for granted that if a PropertyChanged event is fired then the UI will be updated as we will be binding this property using MVVMLight later, so all we need to do is ensure that when we execute the IncrementCommand the correct property change event is fired. FluentAssertions has some nice extension methods to ensure property change events are fired and these take expressions allowing the property in question to be defined in code instead of using a magic string - less fragile if the property name changes.

If you run these tests they should all pass allowing us to have some confidence in this view model.

CountersVieModel

Once we have a view model for the individual counters we can create one for the overall list of counters. For this we will create a new class, CountersViewModel. Like the CounterViewModel above this will also derive from ViewModelBase.

using System.Collections.ObjectModel;
using System.Threading.Tasks;
using GalaSoft.MvvmLight;
namespace StupendousCounter.Core.ViewModel
{
public class CountersViewModel : ViewModelBase
{
private readonly IDatabaseHelper _databaseHelper;
private readonly ObservableCollection<CounterViewModel> _counters = new ObservableCollection<CounterViewModel>();
public ReadOnlyObservableCollection<CounterViewModel> Counters { get; private set; }
public CountersViewModel(IDatabaseHelper databaseHelper)
{
_databaseHelper = databaseHelper;
Counters = new ReadOnlyObservableCollection<CounterViewModel>(_counters);
}
public async Task LoadCountersAsync()
{
var counters = await _databaseHelper.GetAllCountersAsync();
foreach (var counter in counters)
{
_counters.Add(new CounterViewModel(counter, _databaseHelper));
}
}
}
}

This is quite a simple view model - it takes the database helper and exposes a method to load the counters from the database and wrap them all in the appropriate view model. These counters are exposed as a ReadonlyObservableCollection<CounterViewModel>, as opposed to a standard ObservableCollection<CounterViewModel>. Most people just use observable collections to expose collections of objects in the view model, but I prefer to expose them as a read only collection. The reason for this is the public properties and methods expose the intent and capabilities of a class, and to me exposng an observable collection that can be modified suggests that it is perfectly acceptable to modify the collection externally. In this case we don’t want it to be modified externally, and when we implement an add button we will do it via a command that adds the new counter instead of by modifiying the collection directly. Therefore by exposing it as a read only collection we are saying that we explicitly don’t want anyone else to modify it.

Unit testing the CountersViewModel

Despite being a simple view model we still shoudl write tests - always good to get as much converage as possible to catch any silly errors. At the moment all we need to test is that when we load the conters they are loaded correctly so this is a simple unit test class:

using System.Collections.Generic;
using System.Threading.Tasks;
using FluentAssertions;
using Moq;
using NUnit.Framework;
using StupendousCounter.Core.ViewModel;
namespace StupendousCounter.Core.Tests.ViewModel
{
[TestFixture]
public class CountersViewModelTests
{
private Mock<IDatabaseHelper> _mockDatabaseHelper;
private readonly Counter _monkeyCounter = new Counter
{
Name = "Monkey Count",
Description = "The number of monkeys",
Value = 10
};
private readonly Counter _platypusCounter = new Counter
{
Name = "Playtpus Count",
Description = "The number of duck-billed platypuses",
Value = 4
};
private static bool Matches(CounterViewModel cvm, Counter counter)
{
return cvm.Name == counter.Name &&
cvm.Description == counter.Description &&
cvm.Value == counter.Value.ToString("N0");
}
[SetUp]
public void SetUp()
{
_mockDatabaseHelper = new Mock<IDatabaseHelper>();
}
[Test]
public async Task LoadCountersAsyncShouldLoadTheCountersFromTheDatabase()
{
_mockDatabaseHelper.Setup(d => d.GetAllCountersAsync()).ReturnsAsync(new List<Counter> {_monkeyCounter, _platypusCounter});
var vm = new CountersViewModel(_mockDatabaseHelper.Object);
await vm.LoadCountersAsync();
vm.Counters.Should().HaveCount(2);
vm.Counters.Should().Contain(c => Matches(c, _monkeyCounter));
vm.Counters.Should().Contain(c => Matches(c, _platypusCounter));
}
}
}

The single test here will load the counters and ensure they match the ones we’ve mocked the database helper to return. The only lightly new thing here is this test is async - note the async Task return type. This tells NUnit to run these using async/await so inside the tests we can await any async calls, like the LoadCountersAsync method on the database helper.

Exposing our view models

The autogenerated classes provided by MVVMLight expose a main view model and a view model locator. The MainViewModel.cs file isn’t really of any use to us, so it can be deleted. The ViewModelLocator however is useful. It’s an implementation of the service locator pattern using SimpleIoC internally. Once tidied up and changed to suit our needs it looks like this:

using GalaSoft.MvvmLight.Ioc;
using Microsoft.Practices.ServiceLocation;
namespace StupendousCounter.Core.ViewModel
{
public class ViewModelLocator
{
static ViewModelLocator()
{
ServiceLocator.SetLocatorProvider(() => SimpleIoc.Default);
SimpleIoc.Default.Register<IDatabaseHelper>(() => new DatabaseHelper());
SimpleIoc.Default.Register<CountersViewModel>();
}
public static CountersViewModel Counters => ServiceLocator.Current.GetInstance<CountersViewModel>();
}
}

The hordes of comments have been removed as we don’t really need them. The clean method is also gone as again we don’t need it. The important change though is in the static constructor where we register a new DatabaseHelper instance as an IDatabaseHelper, as well as registering the CountersViewModel type. These are registered against the SimpleIoc.Default instance, which is set up as the locator provider for the ServiceLocator. SimpleIoc.Default is a default instance of the SimpleIoc container, and anything registered in a container instance is effectively a singleton inside that instance. Having this default instance set up as the service locators provider means that whenever we access the ServiceLocator.Current singleton and get an instance we will get the single instance that we registered. For example out Counters property gets the instance of CountersViewModel from the service locator, which in turn gets it from the default SimpleIoc instance, which checks if it has an instance of the type and if so returns it, if not it creates a new one injecting any constructor parameters using types also registered in the SimpleIoc instance. If none of this makes sense then I suggest you stop reading here and read up on the dependency injection pattern and service locator patterns. These are pretty standard, allow for nicely decoupled applications, and are pretty fundamental to a good MVVM implementation.

The code for this can be found in GitHub on the Part3 branch at https://github.com/jimbobbennett/StupendousCounter/tree/Part3

In the next part we’ll work on the actual UI and start wiring everything up.

Continuing on from the last post where I mentioned what I was listening to whilst developing, today I'm listening to Heather Nova

Note - these are an affiliate links - if you click them and buy I get a small cut.

Come hear me speak at Xamarin Evolve

Fri, 22 Jan 2016 17:40:47 +0000

If you’re going to Xamarin Evolve 2016 this year then come and hear me talk! I’ll be speaking about using iBeacons and Eddystone in your apps including showing how to use beacons to find a hidden monkey- and giving away a few beacons to anyone who who can find the monkey.

The beacons I will be using will be from Estimote - they make fantastic hardware and software and have pushed beacons above and beyond what they were meant to do to allow you to do some awesome things. If you want to buy beacons, be sure to check them out. I have no affiliation with them, I don’t get anything if you buy their beacons, they just are that awesome that they are worth shouting about.

Building a Xamarin Android app - part 2

Fri, 15 Jan 2016 21:12:13 +0000

This is the second part in the my series about building an Android app using Xamarin.Android. You can find the first part here, and I highly recommend reading that first.

Time to think about data

In the last post we put together the basic app shell, colour scheme etc. Now lets think about the data model, and how we are going to persist the data - it’s no good having a counter if the data is lost every time the app is restarted. For persistence we’ll simply use SQLite as we just need something to be stored locally for now.

The basic set of classes we need are something to represent a counter with its name, description and current value, and a way of storing the dates the counters were incremented. This historical data would allow us to see when counters were incremented - for example it may be nice to see that your ‘cup of coffee’ count is 74 but it would also be nice to show that in a chart showing cups per day instead of just the total.

PCL

Now ‘because we’re not animals’ (phrase ©2016 Mike James) I’ll be putting all the data and business logic into a portable class library (PCL) so that we can easily port this to another platform if required. If you don’t know what a PCL is I suggest you read the docs.

First we create the PCL - I’m going to call it StupendousCounter.Core.

When you create a PCL you have to select which targets it supports so that VS can ensure the right subset of the .Net framework is available. The following targets need to be selected:

Once created we can delete the class1.cs file that is auto generated.

SQLite

Next, we’re going to need to add SQLite. There’s a number of different nuget packages we could add, and a selection of ways we can do this but for this we’re going to use SQLite-net-pcl. Note that this is the SQLite-net**-pcl** version, not the base SQLite-net version which won’t work with a PCL. This nuget package needs to be added to both projects - the .Core and the .Droid.

It’s really easy to create classes that can be stored in SQLite - it can store simple data types out of the box and has attributes to handle fun things like auto-incrementing primary keys.

This is the class we’ll be using for our counters. This is defined in Counter.cs:

using SQLite;
namespace StupendousCounter.Core
{
public class Counter
{
[PrimaryKey, AutoIncrement]
public int Id { get; set; }
public string Name { get; set; }
public string Description { get; set; }
public long Value { get; set; }
}
}

And this is the counter history, defined in CounterIncrementHistory.cs:

using System;
namespace StupendousCounter.Core
{
public class CounterIncrementHistory
{
[Indexed]
public int CounterId { get; set; }
public DateTime IncrementDateTimeUtc { get; set; }
}
}

Note the [Indexed] attribute on the CounterId so that this column gets indexed.

That’s the data classes sorted, now we need to think about how we store and retrieve this data.

Database connection

To help us handle the relationships between the counter and it’s history, and to abstract away the database from the application (good separation of concerns helps with unit testing which we’ll add later), we’ll need to create a database helper. This will contain methods to retrieve all counters, create a new one, update an existing one or increment the counter (and persist the change to the history). We start by creating an interface that defines our helper. It needs to be an interface so that we can mock it out later when unit testing.

Here’s the interface code defined in IDatabaseHelper.cs:

using System.Collections.Generic;
using System.Threading.Tasks;
namespace StupendousCounter.Core
{
public interface IDatabaseHelper
{
Task AddOrUpdateCounterAsync(Counter counter);
Task IncrementCounterAsync(Counter counter);
Task<IEnumerable<Counter>> GetAllCountersAsync();
Task<IEnumerable<CounterIncrementHistory>> GetCounterHistoryAsync(int counterId);
}
}

Notice all the methods are set up ready to use async/await because we’re writing modern code and want to do it properly!

Now lets start constructing the implementation (or to be honest lets get Resharper to create our implementation) in DatabaseHelper.cs:

using System.Collections.Generic;
using System.Threading.Tasks;
namespace StupendousCounter.Core
{
public class DatabaseHelper : IDatabaseHelper
{
public async Task AddOrUpdateCounterAsync(Counter counter)
{
throw new System.NotImplementedException();
}
public async Task IncrementCounterAsync(Counter counter)
{
throw new System.NotImplementedException();
}
public async Task<IEnumerable<Counter>> GetAllCountersAsync()
{
throw new System.NotImplementedException();
}
public async Task<IEnumerable<CounterIncrementHistory>> GetCounterHistoryAsync(int counterId)
{
throw new NotImplementedException();
}
}
}

SQLite is a file based database - it stores it’s data in a single .db file that is stored locally on the device. This location is different depending on which OS it’s running on, so Android puts these files in one place, iOS somewhere different. This means that we have to do some work in the Android project itself to construct the path to the database file, and this path needs to be available to our database helper implementation. We also need to initialise the database to create the tables that will be storing our data. The simplest way to do this is to have a static initialisation method that we call from the MainActivity:

 public class DatabaseHelper : IDatabaseHelper
{
public static void CreateDatabase(string dbPath)
{}

In Android the correct place to store your SQLite database files according to Xamarin is in the personal folder, which is defined quite helpfully in System.Environment.SpecialFolder.Personal. Honestly I’ve yet to find out exactly why this is the recommended place, so if you know please let me know!

In our MainActivity.cs, in the OnCreate method we can grab this folder, add a db file and pass that to the CreateDatabase method on our database helper:

protected override void OnCreate(Bundle savedInstanceState)
{
base.OnCreate(savedInstanceState);
var path = System.Environment.GetFolderPath(System.Environment.SpecialFolder.Personal);
var dbPath = Path.Combine(path, "counters.db3");
DatabaseHelper.CreateDatabase(dbPath);

There are nicer ways of doing this, and we’ll improve this in a later part, so don’t be too offended by this implementation.

For the body of the CreateDatabase we store the db file path, open the database and create all the tables. Luckily SQLite is smart enough to not create the tables if they already exist, so this can be run as many times as you want without worrying about corrupting data.

private static string _dbPath;
public static void CreateDatabase(string dbPath)
{
_dbPath = dbPath;
using (var connection = new SQLiteConnection(dbPath, SQLiteOpenFlags.Create|SQLiteOpenFlags.ReadWrite))
{
connection.CreateTable<Counter>();
connection.CreateTable<CounterIncrementHistory>();
}
}

Now lets implement the main logic. First up, AddOrUpdateCounterAsync:

public async Task AddOrUpdateCounterAsync(Counter counter)
{
var connection = new SQLiteAsyncConnection(_dbPath);
if (counter.Id == 0)
await connection.InsertAsync(counter);
else
await connection.InsertOrReplaceAsync(counter);
}

This method checks the id on the counter - if it’s 0 then its a new counter and is inserted, otherwise it’s updated. Remember our Id field had the [PrimaryKey] attribute, so that’s how SQLite can know which counter to update. For the update we use the InsertOrReplace method, and this checks the Id against the database and if it’s not found then it inserts the item, otherwise it replaces it. Now this sounds a lot like what we’ve implemented ourselves, so why do we do the check for an Id of 0 and call Insert if found? The reason is the [AutoIncrement] attribute. This tells SQLite to automatically increment the id to get the next one - so if we have 10 items in the database and we insert one more it gets its id set to 11. If we insert this counter using Insert then the id on our object gets updated - so we can see the new Id in the Counter instance that is passed in. If we call InsertOrReplace it doesn’t update the instance passed id. I’m not sure if this is by design or a bug but it’s a bit annoying, hence the workaround of calling Insert if the id is 0. This way once this method is called we can always trust the id of the Counter we passed in.

Second up is IncrementCounterAsync:

public async Task IncrementCounterAsync(Counter counter)
{
var connection = new SQLiteAsyncConnection(_dbPath);
counter.Value++;
await AddOrUpdateCounterAsync(counter);
var history = new CounterIncrementHistory
{
CounterId = counter.Id,
IncrementDateTimeUtc = DateTime.UtcNow
};
await connection.InsertAsync(history);
}

This is a little bit more involved. First up it increments the value, then updates the entry in the database to reflect the new value (using our AddorUpdateAsync method to ensure the Id on the Counter is updated). Next we create a history entry, then this history entry gets inserted. Note that the timestamp is stored as UTC time - this allows the times to be adjusted for wherever the user is and if they increment in a different time zone the data doesn’t get out of whack.

Thirdly we need to implement GetAllCountersAsync:

public async Task<IEnumerable<Counter>> GetAllCountersAsync()
{
var connection = new SQLiteAsyncConnection(_dbPath);
return await connection.Table<Counter>().ToListAsync();
}

The generic Table<T> method on the connection returns the table that stores the given type, and from there we get it back as a list of Counters from the database. Just be aware though - these will be new instances. So if you add a Counter then get all the Counter you have added will be a different instance than the Counter in the collection with the same Id.

Finally we need to implement GetCounterHistoryAsync:

public async Task<IEnumerable<CounterIncrementHistory>> GetCounterHistory(int counterId)
{
var connection = new SQLiteAsyncConnection(_dbPath);
return await connection.Table<CounterIncrementHistory>().Where(c => c.CounterId == counterId).ToListAsync();
}

This queries the history based off the counter id.

This is our database connection all finished - so now lets test it. We haven’t wired it up to the UI yet, so lets test it by building some unit tests.

Unit testing our database connection

The best way to verify everything is working is with a stack of unit tests. Normally when writing C# code you create a console application or a class library and create your tests in there. This works well for simple cases, but with SQLite I don’t think it quite cuts the mustard. The problem is the platform specific implementation of SQLite might be different on Android (the platform we’re targeting) and Windows (the platform the tests would be run on). Luckily there is a way to easily run unit tests on Android - a Xamarin Android Unit test project. So lets start be creating a new project of this type.

This project creates an Android app that runs our unit tests. It uses a custom variant of NUnit called Xamarin.Android.NUnitLite, but for all intents and purposes this can be used like NUnit. The app runs like any other Android app and has a ‘Run Tests’ button. This will work through all classes in the project with the NUnit [TestFixture] attribute and run all methods with the [Test] attribute, dumping their output to the screen.

We do have to install a couple of other nuget packages as well - first we need SQLite-pcl-net of course. Second we need FluentAssertions.Xamarin - simply because I love the assertion syntax of Fluent Assertions and this is a version that runs on mobile.

Once all these packages are installed, we can bin the auto-created tests file and create our own called DatabaseHelpersTests.cs.

Writing our first test

Lets start with a simple test to start with which is to test that a newly inserted Counter gets its Id set. In our DatabaseHelpersTests.cs first we mark the class with the [TestFixture] attribute

[TestFixture]
public class DatabaseHelperTests

Then we can create our first test:

[Test]
public void InsertingACounterShouldSetItsIdAsync()
{
var dbfile = Path.Combine(RootPath, Guid.NewGuid().ToString("N") + ".db3");
DatabaseHelper.CreateDatabase(dbfile);
var db = new DatabaseHelper();
var counter = new Counter
{
Name = "TestCounter",
Description = "A test counter"
};
counter.Id.Should().Be(0);
var res = Task.Run(async () =>
{
await db.AddOrUpdateCounterAsync(counter);
return 0;
}).Result;
counter.Id.Should().Be(1);
}

What this test does is create a database with a new unique name (so we can ensure the database is clean), it then creates a new counter and verifies the Id is 0 (love me that assertion syntax!). The next is the odd bit. If you create an async unit test it seems to hang when running it on the simulator, so to work around it I have to run the async code in a Task that is waited for. This works. Very annoying, but not much else we can do. Once the counter is inserted we assert that the counter is updated to 1.

Launching the test app and clicking ‘Run Tests’ will run this test and you’ll see it pass, with ‘1 test passed’ showing on the app.

I’ll add a selection of other tests as well, and these can all be seen in the code on GitHub in the Part2 branch at https://github.com/jimbobbennett/StupendousCounter/tree/Part2.

In the next part we’ll start wiring the UI up to the data using MVVMLight.

Continuing on from the last post where I mentioned what I was listening to whilst developing, today I'm listening to Nicola Benedetti

Note - these are an affiliate links - if you click them and buy I get a small cut.

Building a Xamarin Android app - part 1

Wed, 13 Jan 2016 17:26:15 +0000

I’ve got a bit of time off between finishing my career as a slave to the world of finance IT and starting a new and exciting job as a Xamarin mobile app developer in New Zealand, so I thought I’d try to brush up on my Xamarin.Android knowledge as in the past I’ve mainly focused on iOS.

So to help me do this, I’ve decided to build and publish an app, and document the process. So over the next few posts I’ll be covering the development of Stupendous Counter - the most amazing counter app ever!

Stupendous Counter

The purpose of this app is simple - to count things. Think of it as an automated version of something like this:

Obviously being an app it should be a bit more advanced, so I’ll be adding multiple named counters, count history with fancy charts, location based counts and other funky tools.

Design

First thing to do is to design the UI and flow of the app. I find this a good way to start as it gives me a feel for the views I need to create, the structure of the data I need to store and manipulate and allows me to test the UI out on paper before committing to writing code.

I’ve been playing with a few tools to aid in design, and currently using Fluid UI. It has a free tier that allows wireframe design, which is what I’ve been using. As well as allowing you to wireframe your UI, you can also setup transitions between screens to test out your UI flow.

The design I’ve come up with is a fairly boring, bog standard Android app with all the usual bits - a navigation drawer that’s poor UI, a hamburger menu and a floating action button that does something you don’t do very often.

Here’s a screenshot of the final design for the first version of the app:

And heres the transitions:

For the colour scheme I used Material Palette which is an awesome tool for choosing colours that work well together in a material design app or website. As a traditional developer I’m bad at design and UI so I enlisted the help of my 3 year old daughter to choose the colours - a process that took ages as she was enjoying ‘playing colours on daddy’s computer’.

Let’s get coding

And the best way to write code? Get a tool to do it. To create the basic app structure with styles, navigation buttons etc. you need a lot of boiler plate code. Luckily James Montemagno from Xamarin has done the hard work for us with his Xamarin.Android templates for Visual Studio. These templates provide the boiler plate code for a basic AppCompat app, or an AppCompat app with a navigation drawer.

To start with, I’m going to create a Navigation Drawer App called StupendousCounter.Droid. Note the name ends in .Droid, not .Android - otherwise you get a load of namespace clashes with the Android namespace).

Once the app is created, it looks like this:

First thing to do is to make it more like what I’m after. The App name is wrong, the colours don’t match the scheme chosen by my daughter.

Colours

Setting the colours is easy. In the template there is already a style defined in Resources/values/styles.xml which references colours defined in Resources/values/colors.xml. In the colors.xml file are a set of colour properties that match the output from Material Palette:

These colours can be downloaded as an XML file and the values copied in - you can’t just drop the xml file in as the names don’t write match - for example it’s primaryDark in the template generated code and primary_dark in the material Palette output, but it’s pretty obvious which value to copy into each field. Once done the app looks a bit more like what we’re after:

App name

The App name is in two parts - first the app name in the manifest file which defines the name given in the app settings, second the activity name that appears on the launcher. The app name in the manifest file (Properties\AndroidManifest.xml) is defined as android:label="@string/app_name", so correctly referring to a strings resource file located at Resouces\values\strings.xml. This is the recommended way to handle all text in your app so that you can easily localise it as described on the Android developer docs. Setting the app name is therefore nice and easy, just change the strings.xml to have <string name="app_name">Stupendous Counter</string>. You may need to restart you emulator after re-deploying the app - XAP doesn’t update the name in the settings until you restart.

The main activity is not set up like this though - the template creates it with a hard-coded label of “Home”. I’d like this to match the app name so it’s consistent, so it’s a simple case of changing it to match the manifest: [Activity(Label = "@string/app_name", ...)]. Android uses the @ symbol to indicate a resource, so this syntax will correctly load the activities label from the string resources.

Once done it looks like this:

Done! Now we have a nice navigation drawer based app shell with the right name and the correct colours. In the next part we’ll start adding some functionality. The code for this is on Github here: https://github.com/jimbobbennett/StupendousCounter/tree/Part1 The code in the Part1 branch is the code for this part. The changes we make in later blog posts will be in other branches.

Some useful bits

Here’s a couple of useful tips I cam across when writing this:

Debugging XAP when using a VM

My developer setup is a MacBook Pro running Windows 10 through Parallels and using Visual Studio 2015, with Xamarin Android Player installed as a Mac app, not a windows App (I use it from Xamarin Studio running on the Mac as well, so saving space/download time when upgrading etc. by only having it once). It’s pretty easy to run and debug Android apps on a player running natively on the Mac from VS on a Windows VM - instructions are here.

Big shoutout to Cockos

The animated Gifs in this blog post were captured using LICEcap from Cockos, an awesome free .Gif capture tool.

And as a side line I'll also be listing the music I'm listening to when developing. Right now I'm listening to 'The Shocking Miss Emerald' by Caro Emerald.

Note - this is an affiliate link - if you click here and buy I get a small cut.

Building iBeacon apps using Xamarin.iOS Video Tutorial

Thu, 12 Nov 2015 12:11:07 +0000

As a change from talking about iBeacons at events, I’ve decided to start a series of YouTube videos covering a number of topics around iBeacons.

My first one, an introduction to building your first iBeacon app using Xamarin.iOS is now up on YouTube! You can check it out below.

This is my first one, so would appreciate any feedback.

Free iBeacons

Thu, 20 Aug 2015 18:08:50 +0000

Xamarin Dev Days - Birmingham, 2nd August

Mon, 27 Jul 2015 19:05:31 +0000

Xamarin have announced they are doing their first European Dev Day in Birmingham, hosted by the Birmingham Mobile Cross Platform User Group. This is a great user group organised by Dave Evans, and recently they kindly allowed by to jabber on at them about iBeacons.

Details and tickets are available here:

https://ti.to/xamarin/dev-days-birmingham

I’m very excited by this - the team who will be there are a great bunch, and if you come along you also get the opportunity to meet me! And who would pass up on that ;op I’ll be in town Saturday night through till Monday, so if you’re there and fancy a beer/curry etc. then give me a shout.

Reviewed - Xamarin Mobile Application Development

Mon, 27 Jul 2015 18:03:59 +0000

Xamarin Mobile Application Development

By Dan Hermes (@ lexiconsystems)

Recently I was lucky enough to get my hands on an early draft of Xamarin Mobile Application Development by Dan Hermes. This is a book aimed at the .Net developer who wants to get started with Xamarin, primarily with Xamarin.Forms.

Let me start by saying this isn’t a ‘read it on the train’ book - it’s very code heavy. As each new concept is introduced, you get sparse text with a code example to cement the concepts - so I would say it’s very much aimed at the developer sitting at their laptop who can read and then work through the example code. Luckily the examples are excellent and are available in pure C# code in the book, and in both C# and Xaml on GitHub. Personally I would have preferred if the code focused on Xaml, but then I am a huge Xaml fan. Perhaps a suggestion to the author is to have 2 versions - a pure C# version and a Xaml version (or even 3 and do an F# one).

The book is primarily aimed at iOS and Android development, but it does sometimes dip in to Windows Phone - for example the Xamarin.Forms screenshots and sample code includes Windows Phone, as does some of the cross platform chapters. But this is about the limit of the Windows Phone code - there’s nothing on the platform specific Windows Phone controls for example, wheres iOS and Android controls are covered in detail. Seeing as the market for Windows Phone is so small, this is not a huge omission but Windows Phone developers need to be aware of this.

The book starts off with a brief introduction to cross platform development before diving right into the UI - a sensible place to start and ideal for the new developer as it allows you to quickly get a ‘Hello, World!’ type app running in next to no time. As it introduces the UI’s it discusses the different approaches - native or cross platform. This is an important choice for the developer to make and the pros and cons of each approach are discussed. It then jumps into the controls and provides simple examples of the standard control set available through Xamarin.Forms. This is followed up by describing the same controls from the native platforms, giving a thorough understanding of the widget sets available (although not Windows Phone). These examples are not API level depth - it doesn’t discuss all the properties, methods, events and usage patterns of each control, instead it gives you the basics providing a springboard for your own self discovery. As mentioned before - this is a book for someone who is reading it whilst writing code so can explore the depth of what each control has to offer in their own time.

The controls take up a large part of the book- understandable as there are a lot, and these are the fundamental building blocks of a mobile app. The biggest difference between Mobile apps and other apps you may have been working on is primarily the UI, both the controls and the user interaction patterns so it makes sense to focus a lot on these. Server side code or business logic is the same regardless of platform, so this isn’t really covered here. As well as the basic widgets, navigation patters and their associated control sets are also covered. This is important with a mobile app, as a user needs to navigate using a standard control set.

What is covered next is how your app should integrate with the business logic and back end code, followed by how to do platform specific things in a cross platform code base. I would recommend reading the chapters in a different order to the book at this point though. It covers data access and binding, which is how you app will interact with data and how it can be updated on screen using the standard MVVM design pattern that all Xamarin.Forms apps uses, then talks about platform specific UI before discussing cross platform architectures. My recommendation is to read chapter 9 (Cross-platform architecture) before chapters 7 (Data access and data binding) and 8 (Custom rendered and platform-specific UI), so you can get an understanding of how and why to use the different cross platform techniques before you use them in anger. The coverage of the two different methods, PCLs and shared projects, is good, but I would personally like it to lean heavier on the PCL side - especially as Xamarin themselves are saying this is the preferred method.

TL;RD

Overall it’s a good book for C# coders who want to get into Xamarin for cross platform development. I like Dan’s writing style - it is terse and to the point, but friendly to read. The nearest competitor to this book at the moment is Creating Mobile Apps with Xamarin.Forms by Charles Petzold (affiliate link) and whilst this has the huge advantage of being free and written at the behest of Xamarin (with access to the newest features before they become available), I don’t like the writing style. This is something I’ve always found with Petzold - the guy is a genius and provides excellent materials in his books but they are hard to read with a very dry style. Dan’s book is easier to read. The examples are good, and if you dive into this book willing to code as you read it provides a great introduction.

There are downsides with this book. I would like it to go further - technology moves quickly and Xamarin.Forms is constantly changing including new features like styles and behaviours which are not covered. This is the downside to print, but hopefully the eBook will be able to be updated to cover the latest features, or a follow up will be written. I would also like more on custom renderers as this is a massively powerful way to extent a cross platform app. The basics are covered but this topic is huge. Again, perhaps a follow up book would cover this.
The other downside is the price - Petzold’s book is free and although it’s not as good I can imagine developers dipping into Xamarin for the first time might prefer the free option to get started.

All in all though, it is a good book and well worth buying to work through armed with your favourite IDE in hand. I’m certainly looking forward to any follow ups that Dan writes.

Buy it from Amazon here (affiliate link)

The slides from my iBeacon mini-hack are now available

Thu, 09 Jul 2015 05:45:09 +0000

I had a great time hosting an iBeacon mini-hack with the Birmingham Xamarin Mobile Cross Platform User Group. A big thanks to Dave Evans for organising this.

It was only a small group, which was great as I had enough Estimote iBeacons to give away to everyone. Due to hardware issues and phones not being set up we only got one app built, but everyone seemed to have fun. I’ll be planning a whole day hackathon soon so that more people can try them out and build some cool apps.

I’ve put the slides up on Slideshare and embedded them below.

iBeacons and (lack of) security

Thu, 02 Jul 2015 19:07:05 +0000

So in preparation for my upcoming iBeacon mini-hack I’ve been thinking about iBeacon security. For such a simple device, there is a lot of complex issues around their security to consider.

iBeacons are defined by their UUID and version

A simple statement but with large undertones. An iBeacon is defined by it’s UUID and version - something it broadcasts publicly to the world and is not locked down. There is nothing stoping you reading the id of an iBeacon and setting your app to detect it or your own iBeacon to use it.

Imagine the scenario - UK supermarket Asda are trailing iBeacons to send customers offers and highlight their cheaper prices. The Asda app will monitor for the particular UUID/version combinations (or maybe just the UUID) and when it detects an iBeacon will send a notification to your phone with information. But whats to stop a rival supermarket doing the same thing? All it takes is someone to scan the iBeacons, get their Id’s and register them in the app for the rival supermarket - you walk into Asda and get a notification that the average basket of shopping is cheaper somewhere else. They could even get down to the level of individual offers - if Asda have an offer linked to a particular iBeacon version the rival app could detect the same version and show a price comparison. There is nothing that can be done to stop this - Apple will let you monitor for any UUID, they don’t limit it based on your app. If two apps are listening for the same UUID they will both notify when the iBeacons are detected and nothing can be done to stop it. I have heard that in iOS 8.3 only one app will respond, I haven’t had time to test this yet but in such a case how do you know the right app will respond?

Conversely what is to stop someone depositing iBeacons in a shop that doesn’t support them. If a companies app responds to iBeacons what would stop them from doing guerrilla marketing and dropping iBeacons all around another similar store that causes notifications on their app to guide you away from the store you are in and into their store. Or even environmental activists dropping beacons that send notifications to their app when you use a store that goes against their principles.

A lawyer would be able to provide legal information on this, but I can’t imagine it is illegal to have an app for supermarket A monitor iBeacons owned by supermarket B until a test case comes to court.

iBeacons can be spoofed

Yup - they can be. There is nothing stopping you setting the UUID/version to be whatever you want. iBeacons come with a default UUID so you can connect from the suppliers app and you then update the iBeacon to whatever UUID/version you want. So there is nothing stopping you setting the UUID to match someone else’s iBeacon and causing mischief or mayhem or worse.

Imagine a restaurant uses iBeacons placed on each table to allow their app to locate you at that table. You can order from the app and your food comes over. You then pay from the app - it knows where you are sitting so it knows what bill you are paying. But imagine if someone spoofed the id of your table to match the id of theirs - maybe by sending a much stronger signal from another device. Without realising it you’ve just paid for someone else’s lunch.

Imagine a worse scenario. iBeacons are great in stadiums to help locate your seat. In the event of an emergency a notification can be sent to your phone and based off your location you can be directed to the nearest exit. But in a terrorist scenario, what would stop them deploying iBeacons around so the app cannot locate you, or causes you to be directed towards the threat instead of away. This could lead to loss of life that is not easily prevented.

iBeacons can be hacked

Yup - they can be hacked. The mechanisms used to update the iBeacons from the factory can be used to update them maliciously. This is less of an issue as there are ways to secure access (for example the Estimote cloud), but if they are not secured what is to stop someone walking around a shop and changing all the UUID’s to match a rivals iBeacons so the rivals app is launched instead of the stores app. In these scenarios it’s easy to catch the rival so I can’t image a reputable company doing it - but activists might.

How to get round this?

Estimote offer Secure UUID’s which claim to constantly change the UUID but allow your app to range for the iBeacons despite the UUID changing. I haven’t tried this out yet, but will do so soon and report back. Other than this, there is not much that can be done. I’m hoping Apple will allow you to lock down a range of UUID’s that your app can monitor for to disallow other apps for using them but I guess we will have to see what they do once iBeacon security hits the news.

For more reading about this, I recommend this Fantastic blog post on the Estimote site.

Comparing two mongoDB collections

Thu, 18 Jun 2015 15:55:12 +0000

I’m doing some work for a client at the moment that involves storing files on disk and metadata about those files in a MongoDB instance. The service that manages all this needs to support replication between multiple so that as a file or metadata record gets added/deleted/updated etc., the same changes are applied to all other services that the first on is replicated with. All fairly simple, and quite a normal use case.

As part of my extensive automated testing I needed to be able to compare the resulting metadata after the replication to ensure that nothing is being missed. After a quick look round I couldn’t find anything to help with this, so I decided to write my own at home and open source it.

It’s now on my GitHub page, and on Nuget:

PM> Install-Package JimBobBennett.MongoDbCompare

It’s a simple tool designed to be called from .Net code to compare 2 collections that are based off a known .Net type. You create the class using the type as the generic argument, passing in the connection details to the two MongoDB instances to compare. You then call CompareAsync passing in a function that returns the unique id for each document so that it can marry up the collections and see if they match. The comparison is done by using a simple Equals on each property in each document. Properties marked as BsonIgnore are ignored, as are properties marked as BsonId. You can also provide a list of names for properties that are also ignored (for example I’m using it to ignore a LastAccessed property that stores when the document was last accessed, which is different for each MongoDB instance). The results that come back provide a list of documents that are only in the first collection, a list of documents that are only in the second, and a list of documents that are different. At the moment it doesn’t say which properties are different, but I’ll hopefully be making a change to include this in a later release.

This is not PCL unfortunately, because the MongoDB drivers are not PCL compliant, an only supports .Net 4.5.

Hope it’s helpful for someone. Feel free to raise a PR for anything else you want it to support.

A little less conversation, a little more iBeacons!

Fri, 12 Jun 2015 18:47:03 +0000

iBeacons are a cool technology that allows you to add location awareness to you apps, not based on your location on a map, but based on your proximity to iBeacons, be they at a fixed or moving point. They allow your apps to have an accurate understanding of where you are both inside and out, from sending you offers when you are in a shop to knowing where you are in a sports stadium so you can be directed to your seat. iBeacons are a big up and coming thing - even Facebook are now sending them out to businesses so their app can become location aware. Companies like Estimote are creating and extending iBeacons to bring even more amazing features to the developer.

![iBeacon logo](ibeacon.png)

On the 8th July 2015 I’ll be hosting an iBeacons mini-hack using Xamarin and Estimote beacons, at the Birmingham Xamarin Mobile Cross Platform User Group in Birmingham, UK. In this mini-hack I’ll be talking briefly about this technology and a little about it’s uses, then we’ll hit the code. Bring a Mac laptop, iPhone and lightning cable, and team up with others in building a treasure hunt app using the Estimote iOS Xamarin component and Estimote iBeacons. There will be a prize of an Estimote developer kit or two (containing 3 iBeacons, worth $100 each) for the team that can complete the treasure hunt the fastest.

![Estimote logo](Estimote-1.png)

For more details and to join in visit the Meetup page.

Note - iBeacons are originally an Apple technology. Although Estimote do have limited support for Android we will be focusing purely on their iOS Xamarin component. The iPhone simulator also does not support iBeacons, so you will need an actual device to code against. I will however be splitting the attendees into teams so even if you don’t have an iPhone developer setup come along anyway and I’ll make sure you are teamed up with someone who does.

I'm certified

Thu, 16 Apr 2015 18:53:40 +0000

Just a quick post to share with the world the good news!

After a lot of lectures and a lot of effort I’m now a Xamarin Certified Developer!

Device.StartTimer and iOS backgrounding

Sat, 07 Mar 2015 15:22:21 +0000

I recently completed a lecture on backgrounding in iOS and Android as part of the Xamarin University. During this lecture, one topic of discussion came up with no real answer - how Device.StartTimer() in Xamarin.Forms affects backgrounding. With iOS, if you enable backgrounding using BeginBackgroundTask(), you have 3 minutes to stop all background tasks before the OS kills your app, so I was wondering what would happen if you create a timer using Xamarin.Forms and background the app - will the timer continue or stop.

So I did a little experimenting. I created a new Forms solution, deleting the ‘droid project’ (it would be great to have some more project/solution types available such as forms apps for one platform). I then added the code to enable backgrounding, but commented it out:

public override bool FinishedLaunching (UIApplication app, NSDictionary options)
{
 global::Xamarin.Forms.Forms.Init ();
 LoadApplication (new App ());
 //UIApplication.SharedApplication.BeginBackgroundTask(() => {});
 return base.FinishedLaunching (app, options);
}

Then I added a timer to my Forms app:

public App ()
{
 MainPage = new ContentPage
 {
 Content = new StackLayout
 {
 VerticalOptions = LayoutOptions.Center,
 Children =
 {
 new Label
 {
 XAlign = TextAlignment.Center,
 Text = "Welcome to Xamarin Forms!"
 }
 }
 }
 };

 Device.StartTimer(TimeSpan.FromSeconds(5), () =>
 {
 Debug.WriteLine("Timer tick");
 return true;
 });
}

The timer just logs to the console every 5 seconds so I can see that it is running.

When I tested this out, all was as expected. I open the app, I see the timer ticks, I background the app, the timer stops ticking, I restore the app and ticks resume.

I then uncommented the code to enable backgrounding and tried again. The result is as I expected/feared - the timer ticks, I background the app, it still ticks. After 3 minutes, bang! iOS kills the app.

So be wary - if you are using platform specific iOS code to do backgrounding beware of any Forms timers. Remember to stop them when the app is backgrounded and restart them once it is restored.

The sample code for this is on my GitHub repo.

Xamarin Estimote iOS component

Tue, 24 Feb 2015 21:29:51 +0000

After my Blog post about binding the Estimote iOS SDK, James Montemagno (Developer evangelist at Xamarin) contacted me to say they were about to release an official Xamarin component for accessing Estimote iBeacons and nearables from iOS. This renders my binding library redundant - which is a good thing as its less work for me now to support it!

I decided to have a quick play today, updating the test application that I created to quickly test my binding to use the new component. It was pretty much a one to one swap out, all that needed tweaking were the names. EST was removed from all the names, so things like ESTBeaconManager became BeaconManager. Other name tweaks included events being renamed from DidRangeBeacons to RangedBeacons - nice tidying really that I couldn’t be bothered to do when I bound the library.

The only bad naming I’ve come across so far is their event args classes. Someone left Args on the name of the classes before binding. Remember that the binding project automatically adds EventArgs to the name of the event args class when it is generated, erroring if it already has EventArgs on the end of the name. In this case it looks like someone set it to end in Args, which then gets EventArgs prepended. For example, the RangedBeacons event has the following signature:

public event EventHandler<RangedBeaconsArgsEventArgs> RangedBeacons

Note the rogue Args in the middle.

Other than this naming oddity, it all works perfectly. I’ve only had a quick look but what I’ve seen so far is great. I’m working on an app at the moment which should make a lot more use of the SDK so as I discover things I’ll report back.

One big downside though - it’s iOS only. There is an existing component for Android but this is lacking in functionality. Hopefully Xamarin will do an offical Android component soon.

I’ve updated my GitHub project to use the component, but left the original binding project there for reference. It’s at:

https://github.com/jimbobbennett/EstimoteBinding

The sample project is a Xamarin.Forms project - only with the iOS and PCL projects at the moment until there is an Android component available. It uses the Forms DependencyService to resolve a wrapper around the BeaconManager and Beacon classes from the component. Should be self explanatory but ping me if you want me to explain anything in more detail.

Binding iOS libraries in Xamarin

Thu, 19 Feb 2015 15:25:15 +0000

Update: Since writing this, James Montemagno from Xamarin contacted me to say they are working on an Estimote component. This is now available on the Xamarin component store. Check out his blog post here.

I’ve been playing around with iBeacons a lot recently. They seem a really cool technology with a load of interesting potential use cases. My iBeacon of choice is from Estimote, I’ve got some of their beacons already and have a pack of stickers on pre-order. As well as providing the basic iBeacon functionality, they also provide a whole host of other features including accelerometers and temperature sensors.

To access all their beacon functionality, above and beyond the basics that are part of the Apple iBeacon standard, you need to use their SDK. The Apple CoreLocation API will do the basics - ranging, notifications when going in and out of range etc., but for full access you need their SDK. They helpfully provide it for iOS and Android, but not for Xamarin.

This is not a big deal though, as Xamarin provide an easy enough way to wrap the native SDK’s. This post will focus on how to do it for iOS. I’ll try to do it for Android later and blog about that separately.

Step 1 - Grab the code

The SDK code is on the Estimote GitHub site. Clone this locally.

Step 2 - Create a binding project

Xamarin studio has project types for binding APIs. Start by selecting File -> New Solution, and choosing ‘iOS Binding Project’ from the Unified API section of the iOS entry in the tree. Apple requires all new apps that are submitted to support 64-bit, so to support this in our apps we need to use the new Unified APIs.

This will create a project with a couple of files in it.

ApiDefinition.cs - This file will end up containing the classes created by wrapping the API.
StructsAndEnums.cs - This file will contain any structus or enums that are created from the API.

We now need to add the actual library to the project, so do this by dragging it from Finder/Explorer into the project in Xamarin Studio. Drag the iOS-SDK\EstimoteSDK\libEstimoteSDK.a file into your project. One this is in the project you’ll see it in the tree with an expander by it. Expanding this will show a new file libEstimoteSDK.linkwith.cs. This file contains the instructions on how to link the library, and I’ll cover this in more detail later.

Step 3 - Generate the API

Once we have the library in place, we need to generate the API code to call it. We could do it by hand, but that’s far too much trouble. Instead we’ll use Objective Sharpie, a tool from Xamarin to create the code for you. It’s currently a beta, but works fine for this SDK. I’m hoping one day it will be built into Xamarin Studio so creating binding projects is a one step thing. Objective Sharpie is a command line tool, you give it the header files that come with the library, tell it what SDK to use, which namespace etc. and it will do the magic.

In my case, I run the sharpie command it from the EstimoteSDK folder in the SDK github repository, using the following options:

sharpie bind --namespace=Estimote --sdk=iphoneos8.1 headers/*.h -unified

The options are:

bind - this tells it to bind the header files
--namespace=Estimote - this is the defult namespace used in the created files.
--sdk=iphoneos8.1 - this tells it to use the iOS 8.1 SDK for code generation. You should always use the latest SDK.
headers/*.h - this is the path to the header files you want to generate code for.
--unified - this tells it to use the new Unified API for 64-bit support.

The code generated is in two files - output.cs and output.enums.cs. For some reason it doesn’t output the files using the same naming convention as the binding projects (ApiDefinition.cs and StructsAndEnums.cs). It also doesn’t add using directives to the files.

Once the code has been generated, simply copy the contents of output.cs into ApiDefinition.cs (keeping the namespaces) and output.enums.cs into StructsAndEnums.cs.

Step 4 - Building the code

Not as easy as you may think! If you do a build, there will be errors. This is because the API uses names for parameters etc. that are reserved words in C#.

For example check out the ESTBeaconManager.Constructor method:

IntPtr Constructor (ESTBeaconManagerDelegate delegate);

This has a parameter delegate, which is a reserved word in C#. Luckily, the parameters can be renamed without any issues, so just rename this to del or another name of your chosing. You’ll also have to do the same in ESTBeaconUpdateInfo.

After changing these, build again and you’ll get more errors. These next errors are due to the interfaces in the iOS SDK deriving from NSObject<NSCoding> or NSObject <NSCopying, NSCoding>. The resulting interfaces created by Objective Sharpie derive from NSCoding or NSCopying, and an interface cannot derive from a concrete class. Luckily, we can just remove the all base classes, so interface ESTBeaconVO : NSCoding becomes interface ESTBeaconVO. Do this for all other interfaces that the compiler complains about.

Again, we build, and again, more errors. The next set are due to name casing being changed between iOS and Xamarin - namely NSUUID is NSUuid. A quick Replace all and these are fixed.

The next set of errors are due to missing usings. Objective Sharpie doesn’t add any using directives to the generated code which is a shame. A right click -> Resolve -> using CoreLocation; on CLProximity and Resolve -> using CoreBluetooth; on CBPeripheral and the ApiDefinition.cs file will now build. You’ll need to so Resolve -> using ObjCRuntime; on the [Native] attribute in StructsAndEnums.cs as well.

The code should now build.

Step 5 - Events instead of delegates.

Objective-C has an eventing mechanism based on delegates - you create a class that implements a specific delegate protocol and pass it to another class, which then calls back into the methods of your delegate class when things happen. The code generated by Objective Sharpie follows this pattern as it is a one-to-one mapping to the Objective-C code. Luckily, binding projects are smart enough to be able to translate this into events, with the help of a few attributes to help guide it when it generates these events.

Working down through the code, the first one we come to is ESTBeaconDelegate, used by the ESTBeacon class. The delegate is used to allow the beacon to call back when the connection state changes, or if the accelerometer detects movement.

[Protocol, Model]
[BaseType (typeof (NSObject))]
interface ESTBeaconDelegate {
 // ...
}

[BaseType (typeof (NSObject))]
interface ESTBeacon {
 [Export ("delegate", ArgumentSemantic.Weak)]
 [NullAllowed]
 NSObject WeakDelegate { get; set; }

 [Wrap ("WeakDelegate")]
 ESTBeaconDelegate Delegate { get; set; }
 // ...
}

The code above has been trimmed down to just show the important stuff. The ESTBeacon class has properties to store the delegate - the two different properties and the attributes allow it to strongly type the delegate class that is passed in (Objective-C uses weakly typed delegates).

Rather than use a delegate, we want to use events. This is actually really easy, we mark the ESTBeacon class with an attribute to tell it which properties are used for delegates, and to tell it which delegate types to generate events for.

[BaseType (typeof (NSObject),
 Delegates=new string [] { "WeakDelegate" },
 Events=new Type [] {typeof(ESTBeaconDelegate)})]
interface ESTBeacon {
 // ...

This code tells the compiler to generate events for the ESTBeacon class using the ESTBeaconDelegate to define the events, and the delegate is set in the WeakDelegate property. One of these days I’ll decompile the code to see how it does all this, but I imagine it will create a private class the exposes the delegate and calls back into the ESTBeacon class to raise the events.

We also need to tell the compiler how to generate the event signatures for the delegate. This is also done using an attribute, one that provides the name of the event args class.

interface ESTBeaconDelegate {

 // @optional -(void)beaconConnectionDidSucceeded:(ESTBeacon *)beacon;
 [Export ("beaconConnectionDidSucceeded:")]
 void BeaconConnectionDidSucceeded (ESTBeacon beacon);

 // ...

becomes:

interface ESTBeaconDelegate {

 // @optional -(void)beaconConnectionDidSucceeded:(ESTBeacon *)beacon;
 [Export ("beaconConnectionDidSucceeded:")]
 [EventArgs("BeaconConnectionDidSucceeded")]
 void BeaconConnectionDidSucceeded (ESTBeacon beacon);

 // ...

When the event is created it has the signature:

public event EventHandler<BeaconConnectionDidSucceededEventArgs> BeaconConnectionDidSucceeded;

The class BeaconConnectionDidSucceededEventArgs is created and wraps all the parameters in the delegate call as properties - so it will have public property of type ESTBeacon named Beacon. Note that the attribute doesn’t have EventArgs on the end of the name - this is added automatically by the compiler. If you do add it, you get an error:

BTOUCHTASK: error BI1005: btouch: EventArgs in
EstimoteBinding.ESTBeaconDelegate.BeaconConnectionDidSucceeded attribute should not include
the text 'EventArgs' at the end

This process needs to be repeated for all delegate methods and all classes that use delegates - or as much as you think you will need for internal projects. On my long list of things to do is to look at generating these using VS2015 and see if I can automate this step using Roslyn.

Step 6 - Getting it ready to use with another project

The code now compiles, but as always compilable code isn’t shippable code. There’s a few more things left to do.

First, we need to ensure it supports the 64-bit processors. In the libEstimoteSDK.linkwith.cs you will see it is set to link with v7, v7s and the simulator:

LinkTarget.ArmV7 | LinkTarget.ArmV7s | LinkTarget.Simulator

We need to add the 64-bit processors to this list, so change it to:

LinkTarget.ArmV7 | LinkTarget.ArmV7s | LinkTarget.Arm64 | LinkTarget.Simulator | LinkTarget.Simulator64

Secondly we should review naming. Objective Sharpie tries, but it does get things wrong. For example, the beaconConnectionDidFail method call is named wrong, instead being named by it’s parameter:

// @optional -(void)beaconConnectionDidFail:(ESTBeacon *)beacon withError:(NSError *)error;
[Export ("beaconConnectionDidFail:withError:")]
void WithError (ESTBeacon beacon, NSError error);

This should be changed to be:

// @optional -(void)beaconConnectionDidFail:(ESTBeacon *)beacon withError:(NSError *)error;
[Export ("beaconConnectionDidFail:withError:")]
void BeaconConnectionDidFail (ESTBeacon beacon, NSError error);

I haven’t yet worked out a pattern as to when these odd namings happen - if you work it out please let me know!.

Step 7 - ???

Step 8 - Profit!

Not really, but you now have a nice binding for the Estimote SDK. Hopefully these instructions make sense, tweet me/mail me with any issues you may have or any questions. Xamarin has a load of really helpfull docs covering pretty much everything you would need to know about binding.

And if you can’t be bothered to do all this your self, you can just download my bindings from my GitHub page which I am trying to keep pretty much up to date and includes a Xamarin.Forms based test app to show it all working. I hope to have a NuGet package or a Xamarin component available soon.

JimLib.Xamarin is now updated to the Unified API

Thu, 05 Feb 2015 21:07:19 +0000

Finally, after fighting with Xamarin and having my Surface die and need to be reinstalled twice, I’ve managed to get JimLib.Xamarin updated to use the new Xamarin Unified API so it should be suitable for submitting new apps to the iPhone app store. I’ll be updating my first app Second Screen for Plex soon and sending it to the app store for approval and will update with the results. After all, you never know what weird and wonderful things the app store approval process will turn up.

This is in GitHub, and updated on NuGet.

I'm on iTunes again!

Thu, 29 Jan 2015 20:20:53 +0000

This time for an app, not for a podcast. One of my iTunes apps is finally complete and is now approved and up on iTunes:

Second screen for Plex

If you use Plex as your media server/player then you’ll love this app - it connects to your server and shows you what’s currently playing including the cast and an IMDB link. It saves the hassle of searching IMDB to find out who that actor is or where else you’ve seen that actress. It also allows you to keep tabs on what other people are watching from your server, and it works over the internet (assuming your server is shared via MyPlex) so you can check up to see if the kids are watching cartoons when they should be doing homework.

It’s available for a snip at 79p/99c - less than the price of an extra shot in that expensive coffee you just bought at Starbucks, it’s a no brainer price to make your life so much easier.

I'm on iTunes!

Sun, 25 Jan 2015 21:57:11 +0000

As mentioned before, I wrote a guest post for Core Java Interview Questions all about technical testing as part of the interview process. Sam, the sites editor, has been doing a podcast for the site and was kind enough to invite me along to discuss my blog post with him for an episode. What was meant to be a 20 minute chat turned in to an hour discussing everything from technical tests to the advantages of being an expat.

You can grab it by subscribing to the podcast on iTunes (I’m on episode 4) or direct from the website http://www.corejavainterviewquestions.com/podcast4/.

Xamarin Unified API

Tue, 13 Jan 2015 22:37:52 +0000

Recently Apple announced some changes to their requirements for apps submitted to the iOS app store. From the 1st February all new apps must support 64-bit (for updates to existing apps, it’s 1st June).

Xamarin, not wanting to do anything the easy way have decided that as well as supporting this, they will overhaul their API and unify their Mac and iOS APIs. Based on how poorly they managed the iOS rollout (like not being able to build iOS 8 apps using Visual Studio or on Yosemite until a week after iOS 8 was live), I’m very sceptical about how successful this will be. Combine this with a major update ot Xamarin.Forms at the same time and this is a recipe for disaster. Luckily they decided to have an extra weeks testing and it’s only just been released to their stable channel, with 3 weeks to go until Apple’s deadline.

Tonight I decided to upgrade JimLib.Xamarin. The first thing to note is they have a tool to do the hard work for you. At least they do in Xamarin Studio for Mac - not in the Windows version or in Visual Studio. So the first step in the upgrade is to load it all up in Xamarin Studio on my Mac and run the wizard. So far, so good. This changes the references to monotouch to point to Xamarin.iOS and updates namespaces. Unfortunately if you are using Xamarin.Forms or Xamarin.Settings these referencs stay around, even upgrading their nuget packages to the latest version which supports 64-bit. The easiest thing I found to do was completely remove and re-add these packages. Xamarin.Forms has a nasty thing though - it adds an error condition to the .csproj file if the nuget package is not there, and it doesn’t remove this when removing the nuget package. The fix is to edit the .csproj file and manually remove the error condition. Which is nice.

The next thing to do is fix any type issues. They’ve changed things like int and float (standard C# types) to use nint and nfloat (weird iOS types), which seems non-intuative. Int already changes from 32 to 64 bit depending on platform so I fail to see why they need nint.

The next step is to get it to build. This wasn’t too hard, just cleaning up the mess in the csproj files, changing some types etc.

Finally, I’m tring to build it on Visual Studio. This doesn’t work. I’m getting:

The name InitializeComponent does not exist in the current context

This works on Xamarin Studio on the Mac, obviously, not on Visual Studio.

Will update this once I know more about why…

Update:

I’ve finally got to the bottom of why it was not compiling. To do this I created a new Xamarin.Forms app, added a Xaml file and compared the .csproj files by hand. It seems somewhere on the way the Generator property on the Xaml file was set to MSBuild:Compile. On the new project, it is set to MSBuild:UpdateDesignTimeXaml. I manually updated the .csproj files and it works.

Nasty!

Update 2:

It seems this was documented in the release notes in the Xamarin.Forms forum, but not in the Unified API migration guide. Thanks to Robert Stubbs on the forums for pointing this out. Hopefully Xamarin will improve the docs at some point.

Christmas geekery

Wed, 24 Dec 2014 13:12:46 +0000

Just a quick post to wish you all a very merry Christmas and a happy new year. Or a happy Saturnalia, or Festivus, or however you celebrate at this time of year.

I’ll be spending the time playing with a few things. I’m looking forward to seeing how well Resharper 9 works with Xamarin.Forms, and playing with my Estimote iBeacons. I’ll also hopefuly be recording a podcast for Core Java Interview Questions with my friend Sam, and will be blogging about it all.

See you in the new year!

Technical tests

Thu, 20 Nov 2014 21:15:09 +0000

A good friend of mine, Sam, has started an awesome site for Java developers to help with the always enjoyable process of interviewing.

It’s available at:

http://www.corejavainterviewquestions.com

He was nice enough to ask me to do a guest post, so not being a Java programmer I decided to post about doing technical tests - something nice and language agnostic. The post is question is here:

http://www.corejavainterviewquestions.com/technical-tests/

Have a read and provide any feedback. Always happy to discuss these tests so tweet me at @jimbobbennett with any questions.

Blind technology

Tue, 18 Nov 2014 21:45:01 +0000

Busy, busy, busy. In the last month I’ve relocated form Thailand to the UK with my family, got my UK company up and running (more coming on this in a later post), lined up one client, fought with the Apple app store to get my developer account set up right so I can sell apps (still not done), found a new place to live and been fighting with overbearing bureaucracy. I’ve also found loads of time to spend playing with my daughter before work becomes too overbearing, and the upshot of this is little time for technology. My Udemy course is still nowhere near finished, my apps are not on the Apple app store yet and I don’t even have a working Windows 10 VM (mind you, from what I hear not many others do either).

Hey ho, back to the grindstone from now on.

One cool thing I did have time to do is head to Microsoft’s Future Decoded conference in London. It was a great day, some fantastic speakers (including that drummer guy from D’Ream) and the chance to hear about some awesome technology.

The one piece that really caught my eye surprisingly wasn’t the announcement that Microsoft have open sourced .Net and released new Visual Studio goodies. It was in fact a software/hardware solution from Microsoft Services work with Future Cities in conjunction with Guide Dogs for the Blind to help blind people navigate. A very good friend of mine is almost totally blind so I’m very interested in technology that helps people in his situation. The history of the project is that they were asked to help people navigate around a new city, and seeing as one of their team is blind they decided it would be a good place to start - the needs of the sighted are a lot less than those of people who are visually impaired in a new city so solving for the harder case will also help ensure you cover as much as possible for the easy case.

The basic idea behind the project is to provide visually impaired people with the ability to navigate around a city. For us sighted people it’s easy - we whip out our smartphone and load up google maps. If you’re blind, this is impossible. The Microsoft guys have come up with what is almost like turn by turn navigation using audio clues to guide you around. Dogs are great for stopping you walking out in front of cars but can’t distinguish between Starbucks and a dry cleaners.

The first part of the technology uses binaural sound processing - the ability to make a sound seem like it is coming from somewhere in 3D space using normal headphones. Unlike normal stereo, this allows sounds to be behind, in front, above, or anywhere instead of simply left/right positioned. Combine this with a smartphone with a built in compass and you have the ability to put sound in a particular location regardless of which way the user is facing (assuming the smartphone is facing the same way relative to the user of course).

The downsides for this as a navigation tool are that, as mentioned, the smartphone has to always be facing in the same direction as the user, as well as the user having to wear headphones. A blind person would never want this as their ears are vital to navigating around. If you can’t hear traffic for example then you couldn’t safely walk the streets. The solution to both of these is a special headset. It’s based around bone conductivity headphones (these transmit sound by vibrating the bones in your skull allowing your ears to be free from blockages) with a box of electronics on the back that contains a compass, accelerometer and a bluetooth connection to the phone. This means the headset can always be in the same relative direction to the user to allow the sound to be positioned accurately.

So now we have a tool that can track which way you are facing an provide audio clues at a given compass point. So the next step is to combine this with GPS, map data and a speach interface. For basic city walking it is able to name shops that you are interested in with the name being in the right direction to where you are facing. It can also navigate you to a given destination using a simple sound in the direction you need to turn - so if the place you want is 100 yards ahead on the current street, then right for 50 yards then right again for 20 it will sound in front of you whilst you walk, then turn right when you should turn, then right again when you should turn then announce when you are at your destination. All without being intrusive.

It sounds like such a simple solution and having tried it out I can say it is really well done. I’m hoping they are going to open source the binaural technology so I can have a play at creating something similar.

Good work Microsoft!

Useful links

Wed, 15 Oct 2014 02:44:35 +0000

I thought it would be good to create a page full of usefull links - things that have helped me solve some problems and are complete so there’s no real need for me to blog their content.

Adding symbols to Xamarin crash reports for iOS - this article shows how to add symbol information to a crash report retrieved from your device via XCode for a Xamarin app.

Xamarin.Forms.Labs - Add on controls and features for Xamarin.Forms apps.

Orientation with Xamarin.Forms

Fri, 10 Oct 2014 03:52:28 +0000

Xamarin.Forms is a great cross platform development tool when it works, but being cross platform it suffers from a sever lack of features. Some of these are understandable as they are different on each platform so it’s hard to provide a consistent experience for the developer. Some are bad ommisions as they are vital to all platforms.

The current ommission I’m working on is Orientation. I have yet to see a device that is square. Every device has two orientations, portrait and landscape and a lot of apps change their UI depending on which orientation is in use.

Out of the box, Xamarin.Forms just provides one content for your page, so if you want to change it based on the device orientation you can’t. Some views will handle changes for you - GridView from the Xamarin.Forms.Labs toolkit changes the number of items across based on the orientation, contols will resize based on the available space - but nothing allows a completly different control layout.

So I’ve decided to roll my own. I have my own BaseContentPage class derived from ContentPage to provide some consistent UI features like an activity spinner and services to the ViewModel, and I’ve extended this to include orientation helpers. This is just iOS only for now (if anyone wants to send me a free Xamarin.Andriod business license and Android phone I’ll gladly update it to do ‘droid as well :) )

First I had to create a new renderer for the page to provide information on the orientation and notification of changes.

[assembly: ExportRenderer(typeof(BaseContentPage), typeof(BaseContentPageRenderer))]

namespace JimBobBennett.JimLib.Xamarin.ios.Views
{
 public class BaseContentPageRenderer : PageRenderer
 {
 protected override void OnElementChanged(VisualElementChangedEventArgs e)
 {
 base.OnElementChanged(e);
 ((BaseContentPage) Element).Appearing += (s, e1) => SetOrientation();
 SetOrientation();
 }

 public override void DidRotate(UIInterfaceOrientation fromInterfaceOrientation)
 {
 base.DidRotate(fromInterfaceOrientation);
 SetOrientation();
 }

 private void SetOrientation()
 {
 ((BaseContentPage) Element).OrientationChanged(InterfaceOrientation.GetOrientation());
 }
 }
}

This render calls the OrientationChanged method on the BaseContentPage on creation, on appearing and whenever the orientation is changed by overriding the DidRotate method on the Page renderer. The InterfaceOrientation.GetOrientation() extension method is my own and converts the iOS UIInterfaceOrientation enum into my own Orientation enum to allow it to not be platform specific.

In my BaseContentPage I’ve defined two properties, PortraitContent and LandscapeContent, both of type view. I’ve also changed the default property to be PortraitContent, so when the content is set in XAML it sets the PortraitContent property instead of the Content property. This is done using the ContentProperty attribute, same as in WPF.

[ContentProperty("PortraitContent")]

Internally in my page I set the Content to be a number of views to support my functionality, so when this property is set externally I route the value to a child of one of my grids and reset the content back to the required views. I enhanced this to pass the content to the PortraitContent property to any C# code that sets this will still work. To make the magic happen on orientation change I implemented the OrientationChanged method to set either the only content available if only one value is set, or to set the content depending on the orientation.

if (_portraitContent == null && _landscapeContent == null)
{
 // do nothing
}
else if (_portraitContent != null && _landscapeContent == null)
{
 _contentGrid.Children.Clear();
 _contentGrid.Children.Add(_portraitContent);
}
else if (_portraitContent == null && _landscapeContent != null)
{
 _contentGrid.Children.Clear();
 _contentGrid.Children.Add(_landscapeContent);
}
else
{
 _contentGrid.Children.Clear();
 _contentGrid.Children.Add(Orientation == Orientation.Landscape ? _landscapeContent : _portraitContent);
}

This works well except for one problem. This takes effect after the orientation is changed. What happens is:

Orientation is changed
Screen rotates
Xamarin.Forms re-lays out the controls based on the new orientaion
Content is changed

It looks a bit rubbish as the user sees the portrait content laid out in landscape first before the landscape content is shown. Easy fix though - remove the content before the change, then add the new one after. This can be detected in the rendereer by overriding WillRotate.

public override void WillRotate(UIInterfaceOrientation toInterfaceOrientation, double duration)
{
 base.WillRotate(toInterfaceOrientation, duration);
 ((BaseContentPage)Element).OrientationChanging();
}

protected internal virtual void OrientationChanging()
{
 _contentGrid.Children.Clear();
}

Much nicer.

The only bug I’m left with now that I have yet to fix is for images. I have an image in both my portrait and landscape views that uses the same image source. When changing a few times sometimes the portrait image is blank. I’ve tried raising property changes to fix it but no dice. Once I fix it (or more realistically work round it as it looks like a Xamarin bug) I’ll update this post with the details.

All the code for this is available on in JimLib.Xamarin on GitHub.

Xamarin Insights

Fri, 10 Oct 2014 01:28:54 +0000

At the moment Xamarin are holding their annual Evolve conference in Atlanta. Unfortunately I’m not there, but I’ve been keeping up to date with their announcements and one caught my eye - Xamarin Insights. I’ve been looking for a Xamarin compatible analytics and crash reporter for a while and it looke like this does the job.

So far I’ve loked at:

Flurry - analytics but no iOS 8 Xamarin bindings
HockeyApp - crash reporting but no analytics
Crashlytics - I have no idea what this is as they have a stupid website and a weird invitation process that I can’t be bothered to wade through.

There is no all in one Xamarin compatible solution that I have seen, but Insights seems to be what I’m looking for. It provides event tracing so you can see what bits of your app are being used, crash reporting with upload of dSym files for tracking crashes including seeing the stack and user tracking. It was a bit flakey when I started using it - crash reports wouldn’t show and no user were visible but it improved over the day. I guess Xamarin are hard at work ensuring it’s all working properly as it was only just announced.

It’s pretty easy to set up - just import the NuGet package then call Insights.Initialize("your app id");. After that you can track events using Insights.Track("event name"); and can even pass event data to the track call.

I included it in my latest app beta and it picked up a crash that happened. Unfortunately even though I had uploaded the dSYM file, it wasn’t showing the stack trace. A bit of investigation on my side found the issue - I hadn’t included debug symbols in my build. These have to be turned on so it can link the stack trace back to the lines of code. Easy enough to do in VS.

It’s well worth a try - free to existing Xamarin subscribers during their beta. Hopefully when pricing is announced it’ll be reasonable. I’m also hoping it’ll include distribution as well, it would be nice to upload a build from VS/XS including the dSym file and distribute it to testers and get data back instead of having to also use tools like TestFlight.

iOS pull to refresh without using a UITableView

Sun, 28 Sep 2014 08:38:16 +0000

I’ve been working on the UI for my current app, trying to make some usability improvements. The main screen is a list of data containing an image and some text. Although this is a standard UI pattern, the problem I’m having with it is one of size. List rows are usually short and full width. For text this is fine, but not so good for images, then end up being small and not very easy to see. For the data I’m showing the image is just as important as the text, if not more so for quick identification of the data. I’s also wasting a lot of space as there number of rows will be very small most of the time, probably only one or 2 rows. To improve this I’ve decided to move to a grid like layout, with varying columns depending on if the view is portrait or landscape. The end result is much nicer to look at and use, but coding it up let to a problem.

I was using a derivative of the Xamarin.Forms ListView class with support for pull to refresh courtesy of James Montemagno’s blog. This provides a Xamarin wrapper around the well known UIRefreshControl and tying it to a UITableView. This is all great, until you change from a ListView to something else, in my case a custom generated grid view.

Luckily, UIScrollView also supports pull to refresh, just it’s not as well documented. It has to be a UIScrollView or derivative, not any other control as it needs the pull down bounce to trigger the refresh.

To implement it in my grid view I first created a PullToRefreshScrollView and associated renderer to host my grid. The code for the view lives in my portable project and just defines some bindable properties for the command to execute when refreshing, a flag to turn on the refresh indicator and a refresh message. All lifted directly from James’s PullToRefreshListView.

public class PullToRefreshScrollView : ScrollView
{
 public static readonly BindableProperty IsRefreshingProperty =
 BindableProperty.Create<PullToRefreshScrollView, bool>(p => p.IsRefreshing, false);

 public static readonly BindableProperty RefreshCommandProperty =
 BindableProperty.Create<PullToRefreshScrollView, ICommand>(p => p.RefreshCommand, null);

 public static readonly BindableProperty MessageProperty =
 BindableProperty.Create<PullToRefreshScrollView, string>(p => p.Message, string.Empty);

 public bool IsRefreshing
 {
 get { return (bool)GetValue(IsRefreshingProperty); }
 set { SetValue(IsRefreshingProperty, value); }
 }

 public ICommand RefreshCommand
 {
 get { return (ICommand)GetValue(RefreshCommandProperty); }
 set { SetValue(RefreshCommandProperty, value); }
 }

 public string Message
 {
 get { return (string)GetValue(MessageProperty); }
 set { SetValue(MessageProperty, value); }
 }
}

The next bit is a wrapper for the UIRefreshControl - again lifted from the same place and put into my iOS project.

public class FormsUIRefreshControl : UIRefreshControl
{
 public FormsUIRefreshControl()
 {
 ValueChanged += (sender, e) =>
 {
 var command = RefreshCommand;
 if(command == null)
 return;

 command.Execute(null);
 };
 }

 private string _message;

 public string Message
 {
 get { return _message;}
 set
 {
 _message = value;
 if (string.IsNullOrWhiteSpace (_message))
 return;

 AttributedTitle = new MonoTouch.Foundation.NSAttributedString(_message);
 }
 }

 private bool _isRefreshing;

 public bool IsRefreshing
 {
 get { return _isRefreshing;}
 set
 {
 _isRefreshing = value;
 if (_isRefreshing)
 BeginRefreshing();
 else
 EndRefreshing();
 }
 }

 public ICommand RefreshCommand { get; set; }
}

Finally the renderer. To create the refresh control in the scroll view, it’s a simple case of creating the UIRefreshControl and adding it as a subview of the UIScrollView. When you pull the scroll view down it will trigger a ValueChange event on the UIRefreshControl, which our FormsUIRefreshControl handles to execute the provided command. Now for the gotcha - the refresh only happens when you pull the scroll view down far enough. If the contents of the scroll view is smaller than the available space then it won’t scroll in either direction, stopping the pull action from doing anything. Good news is we can make the scroll view always scroll by setting:

AlwaysBounceVertical = true;

Here’s the full code for the renderer:

public class PullToRefreshScrollViewRenderer : ScrollViewRenderer
{
 private FormsUIRefreshControl _refreshControl;

 protected override void OnElementChanged(VisualElementChangedEventArgs e)
 {
 base.OnElementChanged(e);

 if (_refreshControl != null)
 return;

 var pullToRefreshScrollView = (PullToRefreshScrollView)Element;
 pullToRefreshScrollView.PropertyChanged += OnElementPropertyChanged;

 _refreshControl = new FormsUIRefreshControl
 {
 RefreshCommand = pullToRefreshScrollView.RefreshCommand,
 Message = pullToRefreshScrollView.Message
 };

 AlwaysBounceVertical = true;

 AddSubview(_refreshControl);
 }

 private void OnElementPropertyChanged(object sender, System.ComponentModel.PropertyChangedEventArgs e)
 {
 var pullToRefreshScrollView = Element as PullToRefreshScrollView;
 if (pullToRefreshScrollView == null)
 return;

 if (e.PropertyName == PullToRefreshScrollView.IsRefreshingProperty.PropertyName)
 _refreshControl.IsRefreshing = pullToRefreshScrollView.IsRefreshing;
 else if (e.PropertyName == PullToRefreshScrollView.MessageProperty.PropertyName)
 _refreshControl.Message = pullToRefreshScrollView.Message;
 else if (e.PropertyName == PullToRefreshScrollView.RefreshCommandProperty.PropertyName)
 _refreshControl.RefreshCommand = pullToRefreshScrollView.RefreshCommand;
 }
}

This control is now part of JimLib.Xamarin, available on GitHub.

ComPlexion and legal issues

Fri, 26 Sep 2014 03:11:47 +0000

I’ve been working on an open source Plex REST library. The idea behind it is to provide a simple way to connect to a Plex media server and get info on the currently playing movie or TV show. The most important info being a link to IMDB - pretty much every time my wife and I watch something we dig out of phones to look up an actors name or what else we have seen them in.

The reason I’m developing this library is so that I can use it in an upcoing iOS app that acts like a second screen, providing these details in an easy to use way. More info on this will be coming soon once it’s more complete, but if you’d like to be a beta tester for the app then email me at jim@moomoo.io .

I named my library ComPlexion - simply because it sounded good, included Plex in the name and I’m rubbish at picking names so went for the first one I could think of. However, I decided to have a read of the Plex legal pages with regard to their trademarks just to make sure that I named my app appropriately and email a link to ComPlexion to the Plex legal team just to verify.

As it turns out, they don’t want me to use the name. According to their legal page:

You may:

Name your website, product, or application with something unique, that satisfies the other Guidelines.
Feel free to include language on your site explaining that your application “Works with Plex”.
Use “for Plex” following the name of your application, provided that the name of your application is unique.

You may not:

Use Plex or derivatives thereof in the name of your application

So looks like ComPlexion is out. I’ve renamed it to:

RestAndRelaxForPlex

Not the best name, but it abides by their rules. It’s not a huge problem, they have built a great app and ecosystem so it’s only fair not in infringe on their tradmarks or imply that they are behind or endorse a third party product.

The new GitHub repo is here.

Performance tuning a website

Sat, 13 Sep 2014 10:52:26 +0000

Despite the rise in fast home broadband, tuning your website for performance is still just as important as it was in the days of dial up. Although home connections can be fast in most first world countries (I used to have a 1Gb connection in Hong Kong), there are stil a lot of people in developing countries who only have slower speeds, as well as a huge increase in mobile users who have either limited speed or usage caps (or in my current case, are in a country whose main cables have been cut by a typhoon so the internet is now at dial up speeds until they fix it in a weeks time).

Today as part of working on MooMoo.io I decided to have a look at the performance and see what I could do to improve it. Profiling is important to see what can be improved, but there are some tricks that apply to all websites that should be done as a matter of course. The basic principle is to reduce the amount of data sent over the wire and the number of requests made.

Minification

Minification is the act of taking js or css files and reducing their size - removing comments, white space and other unnecessary characters, renaming variables to smaller names, that sort of thing. The downside to this is it makes the code unreadable, so the normal process is to write JavaScript or CSS normally, then create a minified version (usually named .min.js) as part of a build step. A lot of third party libraries will already provide minified versions of their code, so in general you should always use these where available.

For MooMoo.io I decided to add some code to my Node.js server to automatically minify my CSS and js files during startup, then change my HTML to refer to the minified versions. This means I can develop using the full versions and they are converted when needed. At the same time, I will concatenate all my js files into one. This means the browser only has to make one server request to load all js code instead of one request per file, reducing the bandwidth used.

CSS

For CSS minification, I am using minifier.

$ npm install minifier --save

In my code I already use compass to compile SASS giving me one css file, so I just need to minify that one.

var minifier = require('minifier');

minifier.minify(__dirname + '/public/stylesheets/site.css', {})

This will spit out /public/stylesheets/site.min.css. One quick change to my HTML head code and it’s using the minified version.

For js minification, I am using uglify-js.

$ npm install uglify-js --save

This can take multiple js files and spit out minified code, which can then be saved to a file.

var fs = require('fs');
var uglify = require("uglify-js");

var jsFiles = [
 __dirname + '/../public/vendor/jquery/dist/jquery.min.js',
 __dirname + '/../public/vendor/foundation/js/vendor/fastclick.js',
 __dirname + '/../public/vendor/foundation/js/foundation.min.js',
 __dirname + '/../public/vendor/angular/angular.min.js',
 __dirname + '/../public/vendor/angular-resource/angular-resource.min.js',
 __dirname + '/../public/vendor/angular-route/angular-route.min.js'
];

var uglified = uglify.minify(jsFiles);

fs.writeFile(__dirname + '/public/js/concat.min.js', uglified.code, function (err){
 if(err) {
 console.log(err);
 } else {
 console.log("Script generated and saved:", 'concat.min.js');
 }
});

As mentioned above, uglify creates an object containing the minified code in the code property, so this needs to be saved to a file using fs, the Node file module. In this example, 6 files are concatenated into one minified file not only reducing the soze but also reducing the server hits.

Reducing image sizes

Images can take up a lot of bandwidth. JPEGs have compression built in so reducing the size of these is easy. PNGs are not so easy. They are useful for web sites because they allow transparency, but this comes with a cost of larger file sizes. There are tools to compress PNGs by reducing the colours though. The one I used is TinyPNG - just drop in your images and it spits out compressed versions. I manged to get a 70% reduction in file size across all my images this way.

Compressing resources

The last step is to compress as much as possible when sending data. Modern browsers will request gzipped files where possible to reduce bandwidth. With Node and Express it’s easy to enable gzipping of all content using the Compression middleware.

$ npm install compression --save

This middleware can then be inserted right at the start to compress as much as possible - every request that goes through this middleware will be compressed.

var app = express();
app.use(compression({
 threshold: 512
}));

In this example I’ve set the compression to compress anything below 512 bytes. Below this there is not much point as the zip headers will be bigger than any space you save.

Result

The end result of doing this for me was a much faster page I’ve been using GTMetrix to profile my page and it’s reduced in size from over 600Kb to 200Kb, and is currently rated A by PageSpeed and B by Y Slow - up from F for both when I started.

The only thing I fall down on is browser caching. Seeing as the site is still very mch in development I don’t want to turn this on until it settles down a bit.

FauxGhost - a clean, responsive Ghost theme

Sat, 13 Sep 2014 07:46:23 +0000

This blog, like many others coming out recently, uses the Ghost blogging platform. This is a really nice, simple blogging platform with cheap hosting, built by some super smart people.

The theme support for Ghost is excellent with loads of good themes available (some for money, some open source), and really great documentation on how to create your own theme. Admitedly their marketplace leaves a lot to be desired but it’s early days for the platform so I’m sure this will improve. The default theme Casper is quite nice, but there are better ones. I started off using Ghostion, but since the initial release, no more maintence has been done on it. It also had a few bugs which I needed to fix as well as neededing changes in a few places to add the tweaks I needed, such as links to my twitter feed. Eventually I decided to overhaul it properly and come up with a new theme based off Ghostion.

And so Faux Ghost was born.

Grab it from MooMoo.io. Comments, criticisims, PRs etc. all appreciated.

WeakEvents in PCLs

Sun, 07 Sep 2014 08:05:45 +0000

One of the biggest causes of memory leaks I’ve seen in C# applications is events. Although C# can’t leak in the same way C++ can, it’s easy to unintentionally end up with references you weren’t expecting - and what a lot of people don’t realise or forget is that subscribing to an event holds a reference from the event source to the event target, essentially keeping the target alive.

A simple example is when you have a long lived object:

public class FooManager
{
 public event EventHandler FooEvent;
}

and subscribe from something like a model that is only meant to live for the lifetime of a popup window or other short lived object.

public class ShortLivedModel
{
 public ShortLivedModel(FooManager fm)
 {
 fm.FooEvent += FooHandler;
 }

 private void FooHandler(object sender, EventArgs args)
 {
 }
}

Just by doing this the FooManager internally holds a reference to the FooModel keeping it alive. If you create one model each time the window is popped up you can very quickly build up a big leak, especially if the view model holds onto the model, and the view holds onto the view model - that one model keeps a massic object graph alive.

So how do we deal with it?

The best option is to remember to unsubscribe. But this isn’t always easy. Especially with WPF code there is no simple way to see when a control is closed if it’s not a window. It’s also easy to forget, especially in complicated code where the event subscription could be in a base class you don’t know about or don’t have access to.

There is an easier way - weak events. These don’t hold a reference to the target allowing the target to be GC’d without unsubscribing. Even better , you can use them in the event implementation making it transparent to the subscriber. This is quite a standard pattern and Microsoft provides a good implementation of both a specific and generic weak event manager. The problem with these is that they don’t work for PCL projects - they are too heavily linked to reflection.

Luckily it’s not that hard to create your own that is PCL compliant.

The pattern I’m going to use is to change the event implementation to wire the target of the event into the weak event manager, then tell the weak event manager to fire the event when needed.

Lets start with the calling code first, then build up the weak event manager from there.

public class MyClass
{
 private WeakEventManager _manager = new WeakEventManager();

 public event EventHandler MyEvent
 {
 add
 {
 _manager.AddEventHandler("MyEvent", value);
 }
 remove
 {
 _manager.RemoveEventHandler("MyEvent", value);
 }
 }

 protected virtual void void OnMyEvent()
 {
 _manager.HandleEvent(this, EventArgs.Empty, "MyEvent");
 }
}

In the above code we are declaring a WeakEventManager, then wiring up the value passed to the add handler for the event to it. When we want to invoke our event we tell the WeakEventManager to invoke the event by name with whatever sender and args we require.

Lets look at the implementation now. The full code contains thread safety, a static to get one manager per source object and other helpful bits. The code here is just the basics for brevity.

First, adding the event handlers.

public class WeakEventManager
{
 private readonly Dictionary<string, List<Tuple<WeakReference, MethodInfo>>> _eventHandlers = new Dictionary<string, List<Tuple<WeakReference, MethodInfo>>>();

 public void AddEventHandler<TEventArgs>(string eventName, EventHandler<TEventArgs> value)
 where TEventArgs : EventArgs
 {
 BuildEventHandler(eventName, value.Target, value.GetMethodInfo());
 }

 public void AddEventHandler(string eventName, EventHandler value)
 {
 BuildEventHandler(eventName, value.Target, value.GetMethodInfo());
 }

 private void BuildEventHandler(string eventName, object handlerTarget, MethodInfo methodInfo)
 {
 List<Tuple<WeakReference, MethodInfo>> target;
 if (!_eventHandlers.TryGetValue(eventName, out target))
 {
 target = new List<Tuple<WeakReference, MethodInfo>>();
 _eventHandlers.Add(eventName, target);
 }

 target.Add(Tuple.Create(new WeakReference(handlerTarget), methodInfo));
 }
}

There are 2 AddEventHandler methods to cover events that are of type EventHandler and those of type EventHandler<args>. Both route through to the same helper but are needed as they are not convertible. The code here has a dictionary of a list of handlers and targets to the event name. Each item in the list contains the MethodInfo of the handler (lambdas still have method info so they can be used as the event target) allowing the invocator to call the method, and a weak reference to the target so we know what to call the method on. The weak reference bit is important here. We don’t want to keep a string reference as that would keep the target alive - the thing we’re trying to avoid! For each call we add the passed in handler and a weak reference to it’s target to the list held against the event name. Notice no type safety with the event names. There is nothing that validates that the name given is an event on the source object - the manager doesn’t even know what the source object is! This provides to my mind greater flexibility so the manager can also be used as an event broker.

Next, the event invocator.

public void HandleEvent(object sender, object args, string eventName)
{
 var toRaise = new List<Tuple<object, MethodInfo>>();

 List<Tuple<WeakReference, MethodInfo>> target;
 if (_eventHandlers.TryGetValue(eventName, out target))
 {
 foreach (var tuple in target.ToList())
 {
 var o = tuple.Item1.Target;

 if (o == null)
 target.Remove(tuple);
 else
 toRaise.Add(Tuple.Create(o, tuple.Item2));
 }
 }

 foreach (var tuple in toRaise)
 tuple.Item2.Invoke(tuple.Item1, new[] {sender, args});
}

This code finds the event with the given name in the dictionary, and if it finds it works through the list of handlers. Each target in the list is evaluated to see if it’s null or not. With a WeakReference, the Target returns null if it has been GC’d. There is an IsAlive method but this is pretty useless as there is a race condition, it could be GC’d between the call to IsAlive and the call to Target. The best way is to get a reference to the target, if the target is null the reference is null and if the target is not null the reference stops it being GC’d until we’ve finished using it. If the target is null, we remove the item from the list and carry on. If it’s not null, we store it in a list to invoke at the end. Once we have all the alive handlers, we loop through them invoking the method on the target.

Lastly, we need the remove code.

public void RemoveEventHandler<TEventArgs>(string eventName, EventHandler<TEventArgs> value)
 where TEventArgs : EventArgs
{
 RemoveEventHandlerImpl(eventName, value.Target, value.GetMethodInfo());
}

public void RemoveEventHandler(string eventName, EventHandler value)
{
 RemoveEventHandlerImpl(eventName, value.Target, value.GetMethodInfo());
}

private void RemoveEventHandlerImpl(string eventName, object handlerTarget, MemberInfo methodInfo)
{
 List<Tuple<WeakReference, MethodInfo>> target;
 if (_eventHandlers.TryGetValue(eventName, out target))
 {
 foreach (var tuple in target.Where(t => t.Item1.Target == handlerTarget &&
 t.Item2.Name == methodInfo.Name).ToList())
 target.Remove(tuple);
 }
}

This just loops through the list against the event name and removes the entries for the given target and method name. This is needed to ensure intentionaly unsubscribing from the events works but as mentioned above is not needed to allow the target to be GC’d.

The full code for this is on GitHub and it’s part of the JimLib NuGet package.

Weird goings on with Xamarin

Sat, 06 Sep 2014 07:40:58 +0000

Yesterday and today I had some weird things happening with Xamarin, so I thought it was work documenting them in case anyone else has the same issue.

First - Build errors.

I upgraded to the latest Xamarin and suddenly building for an iPhone using Visual Studio was failing with an odd error:

Failed to resolve "System.Reflection.Emit.ModuleBuilder" reference from "mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089" C:\Program Files(x86)\MSBuild\Xamarin\iOS\Xamarin.iOS.Common.targets

It works on the simulator and from Xamarin Studio on the Mac, but not from Visual Studio. Not a good thing realy seeing as I paid a few hundred dollars extra for my license to enable me to develop in VS.

Luckily someone else had also seen this and a workaround is available. The bug is reported in the Xamarin BugZilla, so check this to see if it’s been fixed. The problem was caused by NewtonSoft.Json. Their .Net 4.5 portable library isn’t that portable it seems, so shows this error. I’ve raised it with NewtonSoft so hopefull yit’ll be fixed. The weird thing here is it only fails when building for the device from VS, not from Xamarin Studio on OXS (I haven’t tried Xamarin Studio on Windows yet). The workaround on Windows is to repoint the dll reference from the portable-net45+wp80+win8+wpa81 version to the portable-net40+sl4+wp7+win8. Unfortunatley, this breaks the build on Xamarin Studio on OSX, so to build there you have to put it back. Repointing the reference involves a change to the .csproj file:

<Reference Include="Newtonsoft.Json">
<HintPath>..\..\packages\Newtonsoft.Json.6.0.4\lib\portable-net45+wp80+win8+wpa81\Newtonsoft.Json.dll</HintPath>
</Reference>

becomes:

<Reference Include="Newtonsoft.Json">
<HintPath>..\..\packages\Newtonsoft.Json.6.0.4\lib\portable-net40+sl4+wp7+win8\Newtonsoft.Json.dll</HintPath>
</Reference>

Second - Weird crashes with StoreKit

This one had me scratching my head for a while. I was using StoreKit for in app purchasing, and when I was loading the list of available products it would crash. Most of the time. No proper error, no debugger breaking, just a crash with a mono SIGSEGV error. When I debugged it it wouldn’t always crash and sometimes it would just work. It was the kind of weird thing that I would normally contribute to a race condition but there wasn’t any threading issues that I could see. In my frustration after a couple of hours I did what I should have done at the start - search StackOverflow. One quick search and I had my answer. When making a request, the request has to stay alive long enough for the callbacks. If you declare it as a local variable like I was doing it can be GC’d before the callbacks. The non deterministic nature of the GC was why it didn’t always happen.

My code was:

public void LoadAvailableProducts(params string[] productIdentifiers)
{
 var productKeys = new NSSet(productIdentifiers);
 var request = new SKProductsRequest(productKeys);

 request.RequestFailed += (s, e) => { /* Do something */};
 request.ReceivedResponse += (s, e) => { /* Do something */};
 request.Start();
}

Notice the local variable request? That was my culprit. One change to a field and it all works:

private SKProductsRequest _request;

public void LoadAvailableProducts(params string[] productIdentifiers)
{
 var productKeys = new NSSet(productIdentifiers);
 _request = new SKProductsRequest(productKeys);

 _request.RequestFailed += (s, e) => { /* Do something */};
 _request.ReceivedResponse += (s, e) => { /* Do something */};
 _request.Start();
}

NDepend - Part 1

Fri, 29 Aug 2014 09:43:13 +0000

Patrick from NDepend reached out to me today to ask me to review NDepend. Having dabbled with it a while ago, I was more than happy to say yes and try out the latest version.

If you’ve never heard of NDepend before, it’s a code quality tool. It analyses your code and provides a number of quality metrics such coupling and cyclomatic complexity - check out all the features here. These are a good way to get an overview of the general quality of your code - it doesn’t tell you if your code works but gives a view on how well coded your application is. For example, a high cyclomatic complexity means you have a lot of code paths - this doesn’t mean your code is broken but it does mean it will be harder to read and maintain. And as every developer knows, the largest portion of your job is maintaining code, so we want it to be as easy as possible.

I like having these kind of metrics available, along with measurements like unit test count and coverage. What I look for in these is both the numbers as well as trends. If the complexity is going down, the unit test count is going up and the coverage is staying high or increasing then I know I and my team are working to keep the quality up or improving. Any negative trends mean that things are potentially getting worse so things need to change. These kind of tools are great locally so you can monitor your code before check in, but more important as part of continous integration and delivery - allowing you to monitor the trends across builds and even block check ins or fail builds if metrics move too far in the wrong direction.

NDepend works as a stand alone tool, a Visual Studio plugin and with a build server. One big downside for me is it doesn’t run on the Mac - I’m developing iOS apps in .Net using Xamarin so need my build agent running on a Mac. So lets start the review from the beginning - installation.

The ‘installer’ is a zip file, so you can just dump it anywhere. This makes it easier in corporate environments with locked down admin. Once unzipped and the license file dropped in, I started with the Visual Studio add-in as my prefered workflow is to do everything from VS before check in. I closed all Visual Studio instances that I had open, ran the add in installed, told it to install into VS 2013. I then launched my first solution - a Xamarin.Forms app and the NDepend menu was there. I then loaded a second solution to help me test my app - this one is a Node.js solution that I use for my apps back end. As it’s loading I get this:

Not a good start. I had to close 8 of these then wait a minute before VS became responsive. I closed and tried again - the same. I then closed all instances of VS and tried again - same same. I then tried another Node solution and this worked. Seems it only happens with my main app backend soluton.

Obviously I can’t carry on like this, so I’ve sent the details to NDepend support and uninstalled it. NDepend does provide a standalone tool for doing this analysis, but my preferred workflow is to do all my unit testing/coverage checking/static analysis inside Visual Studio before check in.

Update

To me a good measure of a person or company is not if they make mistakes but how they deal with them. In the case of NDepend they have been excellent. Their support team have been working with me to help address this issue - including making a good effort to try to reproduce the issue with no success. My code contains proprietry IP so I couldn’t just send it to them, so I’ve been trying to reproduce the issue in something I can send. Turns out, I can’t reproduce it either. It only happens with my Node soution run from one folder - if I copy the whole folder somewhere else it works. In the end the fix was simple - delete everything and re-clone from git. Once I did this, everything worked for a while - then VS crashed and the error returned. It does seem as there are issues with NDepend and Node projects including crashing when right clicking on folders. Luckily the Node tools team are on it and have addressed it for a future release.

Once I’m up and running I’ll be putting NDepend through it’s paces with JimLib and adding more posts.

The features described and my opinions are based on my time spent with NDepend. If anything is incorrect please feel free to correct me and I will update this review

Setting the font for an Entry control using Xamarin.Forms

Wed, 27 Aug 2014 05:02:19 +0000

Xamarin.Forms depsite being very cool is stil a bit lacking when it comes to not only a good range of controls, but also to the abilities of those controls. One example of this is the Entry control - a simple text box. One thing you can’t do with the out of the box implementation is to set the Font. Such a normal thing to do, but not supported.

Luckily, it’s reasonably easy to roll your own controls. Due to the platform independent nature of Xamarin.Forms, to create a control you have to actually create 2 pieces - a portable control, then platform specific renderers to draw the control on each device. I don’t have an android subscription, and can’t be bothered with WinPhone at the moment, so will only be showing how to do this for iOS.

First, we need the control inside a portable project. These can be derived from an existing control - here we derive from Entry as we want to extend its functionality.

public class ExtendedEntry : Entry
{
}

Then we add the bindable properties. These are Xamarins version of dependency properties - they have a static field declaring the property then an instance property to get/set the value.

public class ExtendedEntry : Entry
{
public static readonly BindableProperty FontProperty =
BindableProperty.Create("Font", typeof(Font),
typeof(ExtendedEntry), new Font());
public Font Font
{
get { return (Font)GetValue(FontProperty); }
set { SetValue(FontProperty, value); }
}
}

This is all we need for our control. Next up is the renderer. This wraps platform specific controls to provide the necessary functionality. In the case of the Entry control on iOS, the UITextField control is wrapped. To set our font we need to pass the value from the control to the UITextField in the renderer.

First we create our renderer deriving from the existing entry renderer:

public class ExtendedEntryRenderer : EntryRenderer
{
}

The we override a few methods - OnElementChanged to handle the creation of the control and OnElementPropertyChanged to handle any updates to the properties on the control. In these methods we need to set the font on the UITextView. The Control property of the renderer refers to the underlying UITExtField, the Element property is the ExtendedEntry control.

public class ExtendedEntryRenderer : EntryRenderer
{
protected override void OnElementChanged(ElementChangedEventArgs<Entry> e)
{
base.OnElementChanged(e);
var view = (Labs.Controls.ExtendedEntry)Element;
SetFont(view);
}
protected override void OnElementPropertyChanged(object sender, PropertyChangedEventArgs e)
{
base.OnElementPropertyChanged(sender, e);
var view = (Labs.Controls.ExtendedEntry)Element;
if (string.IsNullOrEmpty(e.PropertyName) ||
e.PropertyName == "Font")
SetFont(view);
}
private void SetFont(Labs.Controls.ExtendedEntry view)
{
UIFont uiFont;
if (view.Font != Font.Default &&
(uiFont = view.Font.ToUIFont()) != null)
Control.Font = uiFont;
else if (view.Font == Font.Default)
Control.Font = UIFont.SystemFontOfSize(17f);
}
}

As you can we, the Xamarin.Forms Font class provides a nice converter to UIFont to make our life easier, and if the font is not supported we resort to a default system font and size. For the property change we just update the font if the property that changes is the font, or if a property change is raised for an empty string - this saves updating when not required and the empty string check is because raising with an empty string as the property name is the way to say all properties have been updated.

If we keep it like this, then the font is changed, but there is a bug - the height of the control doesn’t change to match the height of the font, we make the font larger and the top/bottom of the text is missing. The final step is to set the height. Luckily UITextField will work out the required height based on the font that is set, so we can create a dummy UITextField, set it’s font, get the required height and apply it to both our control and the UITextField hosted in it:

private void ResizeHeight()
{
if (Element.HeightRequest >= 0) return;
var height = Math.Max(Bounds.Height,
new UITextField
{
Font = Control.Font
}.IntrinsicContentSize.Height);
Control.Frame = new RectangleF(0.0f,
0.0f,
(float) Element.Width,
height);
Element.HeightRequest = height;
}

This method uses the HeightRequest property of the Entry control to request a large/smaller height as necessary. This causes the layout to adjust the size to fit if possible. If the HeightRequest is already set (it is -1 if not set), then we do nothing as the user has already decided on what height is needed. All we need to do is just call this after setting the font and everything now works.

The full source for EnhancedEntry including other properties is available as part of JimLIb.Xamarin. I’ve also raised a PR to add this to Xamarin.Forms.Labs.

Continuous deployment with JimLib

Wed, 27 Aug 2014 03:41:57 +0000

Today I’ve finally moved to the world of continuous deployment - albeit for one of my projects only so far, but it’s a start. For my JimLib open source API I’ve automated the whole deployment process so after checkin it builds and deploys to NuGet automatically.

Steps in the process

I do all my development in Visual Studio, which now can connect to git. I have this wired up to my GitHub repo so I can commit changes to my local repo and sync back to GitHub.

Once changes are synced, I use AppVeyor to do my builds. They provide a free service for public repositories so can automate all open source builds without you spending a penny. I have set this up to:

Pick up pushes to my repo
Increment a version number and set this in AssemblyInfo.cs
Build everything
Run all unit tests
Package up the NuGet package including symbols
Push the NuGet packages to NuGet.org

AppVeyor even provides markdown for a build status:

How can I be sure I’m deploying the right thing?

This is where unit tests come in. I try to get as much coverage as possible (91% at the moment), and use this to provide a level of certainty that any new changes work (assuming my tests are good) and that I haven’t broken anything with any new changes. The build process runs the tests and won’t push the package unless all tests pass.

Downsides to AppVeyor

AppVeyor is very good, I’m really impressed with what they offer for free. It’s not fast to get a build, sometimes you can be waiting for up to an hour but for free thsi is fine. The only downside so far is NuGet package restore. They always provide a clean environment to build on and even with package restore turned on you have to build twice with they don’t do. It means you have to check your packages into git. Not the end of the world, but a bit annoying.

Securing a Heroku site with SSL

Sat, 16 Aug 2014 10:12:53 +0000

For my mobile app development work, I’ve been building a Node.js website with a REST api that my apps can use. It’s hosted at MooMoo.io and is running using a free Heroku account - I don’t have anything useful on my site or many users of my API at the moment, so their free single dyno is perfect for my needs. I’m also taking advantage of the free sandbox MongoDB account from MongoLab. It’s pretty cool just how much I can get for free, and I only need to pay if my apps take off and I start making money.

One thing I did have to pay for though is SSL. For my API I don’t want unencrypted connections, so the easiest thing to do is to access it only over HTTPS. It took me a while to get it all set up, so I thought it would be usefull to document the steps in case anyone else is doing the same thing.

First, you need an SSL certificate. My domains are currently with DNSimple, a great company that tries to be simple and honest and not full of the crap that other sites have. Assuming you have a DNSimple account holding your domains (and if not, transfer them already!), it’s simple to get a certificate from RapidSSL with one click and $20. All it takes is an approver with an email address that matches your URL (google apps provides 30 days free trial if you don’t have email set up) and you get the certificate. Once you have the certificate, you can view it and the private key on DNSimple - and copy them into 2 files, server.crt for the certificate and server.key for the private key. For RapidSSL you do need an additional certificate downloaded from their site, but this is easy to get with curl:

curl https://knowledge.rapidssl.com/library/VERISIGN/INTERNATIONAL_AFFILIATES/RapidSSL/AR1548/RapidSSLCABundle.txt > bundle.pem

Next, upload the keys to Heroku. You need to enable the SSL Endpoint add-on - This is not free - it’s $20 a month so it’s not cheap. To upload, use the Heroku toolbelt:

heroku certs:add server.crt bundle.pem server.key --app <your app name>

This will then change the url to your app, so you’ll need to update the CNAME record for your custom domain if you are using one to point to the new url. The output of the certs:add command will show the new url:

Resolving trust chain... done
Adding SSL Endpoint to <your app name>... done
<your app name> now served by <something>.herokussl.com
Certificate details:
...

Heroku implements HTTPS in an interesting way - the connection to Heroku is encrypted and converted to a raw HTTP request that is forwarded to your app. This means that your app doesn’t need to handle HTTPS, just HTTP which makes it easier to develop. The downside is Heroku has access to the unencrypted data, but with any cloud provider there has to be trust.

The last step if you want to do it is to redirect from HTTP to HTTPS. Normally you could use req.secure in Node to identify secure requests, but this doesn’t work with Heroku as all requests to your app are unencrypted HTTP. Luckily, Heroku adds some custom headers to the request that allows you to identify if the original url was HTTP or HTTPS. x-forwarded-proto is used to indicate the protocol that was used for the request that was forwarded to your app. If this is set to https, you know the request was from HTTPS. If not, you can redirect:

if (env != 'development') // only redirect in prod
{
app.use(function(req, res, next)
{
if (req.headers['x-forwarded-proto'] != 'https')
res.redirect(['https://', req.get('Host'), req.url].join(''));
else
next();
});
}

To see this in action, head to http://www.moomoo.io, and you’ll be redirected to https://www.moomoo.io.

JimLib.Xamarin now available

Fri, 15 Aug 2014 07:30:06 +0000

My second NuGet package is now available: JimLib.Xamarin. This contains a selection of helpers to make developing iOS apps with Xamarin easier (WinPhone/Android will come later when I have an Android license/Android device/Windows Phone device to test on).

It’s all open source - code is on GitHub, symbols are also available (as talked about here).

Grab it and give it a spin. Contributions/criticism/pull requests/money/beer always welcome.

Grouping in ListViews with Xamarin.Forms

Fri, 15 Aug 2014 03:38:39 +0000

Xamarin.Forms provides a lot of cool features to use for mobile app development, and the documentation is pretty good - but sometimes the cool features you want to use are a bit lacking in docs. There are example projects but it can be hard to interpret what you need to do to get something working.

The latest area that was confusing me was grouping in ListView. There are properties for specifiying the grouping, but no details on the correct way to use it. After a bit of trial an error, I managed to work out the following:

Your ItemsSource should be a collection of something derived from a collection - each of the inner collection objects is a group (e.g. for people grouped by first initial you would have a maximum of 26 items in your outer collection - each item being a collection of people with the same first initial).
The GroupDisplayBinding property should be set to a property on the inner collection objects - so rather that have a collection of collections, you need a collection of something derived from collection that adds a property to provide the group name.
If you want fast scrolling you need to provide the GroupShortNameBinding property to bind to the short name (e.g. first letter of a name).
If you want more magic in the group header, set the GroupHeaderTemplate instead of the GroupDisplayBinding to have a custom header (setting GroupDisplayBinding clears GroupHeaderTemplate, so you can only set one or the other).

Here’s an example of the kind of classes you can use for the ItemsSource property:

public class InnerCollection : List<string>
{
public string Title { get; set; }
}
public class ItemsSourceCollection : List<InnerCollection>
{
}

Then use this like so:

public class MyPage : ContentPage
{
public MyPage()
{
var listView = new ListView();
var items = new ItemsSourceCollection();
listView.ItemsSource = items;
// bind the grouping to the title of the inner collection
listView.GroupDisplayBinding = new Binding("Title");
}
}

Rather than create this structure each time, JimLib contains a handy ListItemCollection<T> class to provide all this functionality and more.

Symbols for NuGet packages

Fri, 15 Aug 2014 03:11:18 +0000

Since releaseing JimLib onto NuGet, I’ve been dogfooding it as much as possible for my own development. One area I noticed was lacking was in symbol support - if I wanted to view the inner workings of my classes I had to decompile the source using ReSharper. Not an ideal situation.

Luckily there is a better way. SymbolSource is a free service that hosts symbol and source packages for a huge array of NuGet packages and it’s integrated into Visual Studio - if a symbol package is available there then with one click you can navigate the source and even step through the code in the debugger. And if that’s not great enough as it is, you can easily create and upload symbol packages right from NuGet with minimal changes. The docs are here but basically you add the pdb files and source to your .nuspec, then let NuGet do the magic for you - it will create 2 .nupack files when you pack with the -symbols flag, one without symbols with the normal name, and one with with an extension of .symbols.nupkg. When publishing there is nothing extra to do - if the .symbols.nupkg package is found it will be uploaded to the SymbolSource server at the same time as the main package is uploaded to NuGet.

Check it out - grab the latest JimLib NuGet package, use it in your app and step into the code.

Images for an iOS app

Thu, 14 Aug 2014 02:18:56 +0000

When developing for iOS there are a number of different image sizes needed. Just for the springboard icon you need different sizes for iPhones and iPads, both retina and normal versions. When you add spotlight icons (again both retina and normal), iTunes icons the list gets huge.

To help with this and to practice my Node.js skills, I created a node app that does it for you. It’s based on the default express app, you give it an image file and it downlods you a zip file containing all the possible image sizes you could want all named correctly. The code uses Express 3. I know express 4 is out, but 4 doesn’t support file uploads using body-parser. When I work out how to do it, I’ll update the code.

This is built using the Node.js tools for Visual Studio but the code will run with any node install. You will need ImageMagick installed to make it work.

Clone it from my GitHub

Todays tea

Thu, 14 Aug 2014 02:11:16 +0000

Todays bubble tea is banana milk tea with bubbles. Quite nice, but a bit too sweet and the banana taste is a bit artificial.

I also snapped a quick photo of my lunch yesterday being cooked fresh for me. One of the great things about thai street food is a lot of it is cooked in front of you so you know whats in it and that it’s really, really fresh.

Bubbles, bubbles, bubbles. My bubbles!

Mon, 11 Aug 2014 02:25:02 +0000

More bubble tea today. Coffee milk tea with bubbles. Tastes of sweet coffee with an aftertaste of tea - weird but not unpleasant. Nice chewy bubbles.

My first NuGet package

Sun, 10 Aug 2014 03:34:31 +0000

For pretty much every project I’ve worked on I’ve used the same set of extensions and helper classes to make my life easier. For example:

Fluent API on strings myString.IsNullOrEmpty() instead of string.IsNullOrEmpty(myString).
An ObservableCollectionEx<T> class that allows adding multiple items but only raising on CollectionChanged event.

For my latest project, I’ve found myself creating these all over again. Seeing as I’m using this for an open source API, I thought it might be time to wrap these up into another open source library and make it available on NuGet. Therefore I present JimLib to the world.

The source is on GitHub with the API described on the Wiki.

The package is available from NuGet.

Install using Install-Package JimBobBennett.JimLib.

Submissions and feedback are welcome. I’ll be expanding this library as I go along.

Bubble Tea

Sun, 10 Aug 2014 02:20:45 +0000

After a post on FaceBook about enjoying some Bubble Tea, a friend of mine suggested that I should blog about it.
Seeing as I am currently in Thailand, home of the best food in the world, I guess it makes sense to extend this to all food. Most of my meals are incredible food cooked fresh and purchased from markets for usually 40p-80p a dish - so why not share them with the world.

I’ll start today with bubble tea - read about it on Wikipedia if you’ve never heard of it. Essentially it’s a Taiwanese drink made of milky tea with tapioca pearls in the bottom, but there are many variants made with green tea, fruit teas, juices and jellies instead of bubbles.

This morning I stopped at the Bubble Boy (a chain of Bubble Tea stands) at the Ekkami BTS station.

I grabbed a Chocolate milk tea with chocolate pudding at the bargain price of 45 baht (83p/$1.40). It’s really, really nice - you can’t taste the tea that much, just a milky chocolate drink with undertones of bitter dark chocolate. The chocolate pudding is dark chocolate flavour pearls that are nice and chewy without a too intense taste. Definitely one I’ll have again.

Creating a NuGet PCL that works with Xamarin.ios

Thu, 07 Aug 2014 06:33:27 +0000

I’ve been playing with Xamarin.Forms recently to develop an iOS app. The aim is to use my portable open source Plex API in the app (more details of the app will come later once it’s ready to release).

The basic concept of Xamarin.Forms is to have a core portable class library containing all the application code, and very thin platform specific libraries that wrap the core code in an application that targets the relevant platform.

In principle - all very easy. The problem comes when you want to use other libraries with it. These libraries must be portable class libraries - PCLs. Again in principle, very easy. But the devil is in the details…

The main stumbling block I hit was HttpClient. The Microsoft PCL I use to access the Plex REST API. In principle this is a PCL so should work on all platforms. But it doesn’t. The issue is actually with Xamarin.ios. This doesn’t support the Microsoft HTTP client, instead it uses the Mono version. This means my Plex API has to target both the MS and Mono versions - which is a problem as the namespaces are the same, and my iOS app project can’t have a dependency to the MS version, and the Windows Phone project can’t have a dependency to the Mono version.

My solution was to do the following:

Change my code library code to not access the HTTPClient directly, but have an interface to a wrapper class.
Create a Xamarin.ios class library project that has an implementation of the interface using the Mono HTTPClient library.
Create a Windows PCL project (Windows 8, Windows Phone and Windows Phone Silverlight) that links to the source of the iOS implementation (the namespaces are the same so the source can be shared) that references the MS PCL HTTPClient NuGet package.

This will spit out 3 dlls - one PCL for the core with no HTTP access, an iOS class library using the Mono HTTPClient and a Windows PCL that uses the MS HTTPClient.

Next challenge was to get this into a NuGet package. The more recent versions of NuGet have full PCL support which makes the whole thing easier. To install the dlls into the correct targets I just had to ensure the target of the files was set to the correct location:

<files>
<!-- portable -->
<file src="..\Complexion.Portable\bin\$Configuration$\Complexion.Portable.dll" target="lib\portable-net45+win+wp80+MonoAndroid10+MonoTouch10\Complexion.Portable.dll" />
<!-- ios -->
<file src="..\Complexion.Portable\bin\$Configuration$\Complexion.Portable.dll" target="lib\MonoTouch10\Complexion.Portable.dll" />
<file src="..\Complexion.ios\bin\iPhone\$Configuration$\Complexion.ios.dll" target="lib\MonoTouch10\Complexion.ios.dll" />
<!-- Win -->
<file src="..\Complexion.Portable\bin\$Configuration$\Complexion.Portable.dll" target="lib\portable-net45+win+wp80\Complexion.Portable.dll" />
<file src="..\Complexion.Win\bin\$Configuration$\Complexion.Win.dll" target="lib\portable-net45+win+wp80\Complexion.Win.dll" />
</files>

The core portable library is output to a fully portable target, as well as to individual targets for iOS and Windows. The individual platform specifi dlls are then output to specific targets. Using this setup, if I install this package to a portable project, just Complexion.Portable is installed. If it’s for iOS (MonoTouch) then Complexion.Portable and Complexion.ios is installed. Similarly for Windows 8/Windows Phone Complexion.Portable and Complexion.Win gets installed.

The final addition was dependencies. For the portable dll there are no other dependencies. For ios there are also no additional dependencies as the Mono HTTPClient is available in the default Xamarin.iOS references. Windows is different though - it needs the Microsoft HTTPClient NuGet package installed. Thankfully NuGet now supports dependencies by different targets. Groups can be defined with a target attribute and these are only installed into projects that match the target. This means I can add groups for the different windows targets that have the Microsoft HTTPClient dependency, and nothing will be installed for other platforms.

<dependencies>
<group targetFramework="net45">
<dependency id="Microsoft.Bcl" version="1.1.9" />
<dependency id="Microsoft.Bcl.Build" version="1.0.14" />
<dependency id="Microsoft.Net.Http" version="2.2.22" />
</group>
<group targetFramework="win">
<dependency id="Microsoft.Bcl" version="1.1.9" />
<dependency id="Microsoft.Bcl.Build" version="1.0.14" />
<dependency id="Microsoft.Net.Http" version="2.2.22" />
</group>
<group targetFramework="wp80">
<dependency id="Microsoft.Bcl" version="1.1.9" />
<dependency id="Microsoft.Bcl.Build" version="1.0.14" />
<dependency id="Microsoft.Net.Http" version="2.2.22" />
</group>
</dependencies>

Check out my GitHub project for the full API code nuspec file.

Complexion - an open source API for Plex

Mon, 04 Aug 2014 02:46:47 +0000

I’ve just released the first alpha version of Complexion, available on my GitHub page. This is my first true open source API, which I hope to put up on NuGet in the near future.

This is a portable .Net API that connects to Plex - either to a local Plex Media Server specified by it’s IP or hostname, or via MyPlex to discover your media servers. This does require a PlexPass subscription.

To use the API via MyPlex, create an instance of the MyPlexConnection and connect using your MyPlex username and password to get all supported devices and servers that use your MyPlex account.

var myPlexConnection = new MyPlexConnection();
await plex.ConnectAsync(<userName>, <password>);

From each device, you can connect to a Server using the Server class. You can also connect using a local URI if you don’t want to connect to MyPlex.

var myPlexServer = myPlexConnection.Servers.First();
var server = new Server(myPlexServer, <userName>, <password>);
await server.ConnectAsync();

var server = new Server("192.168.1.1"); // IP address of local server
await server.ConnectAsync();

From here you can get a list of videos now playing on each client:

var videos = await server.GetNowPlayingAsync();
foreach (var video in videos)
Console.WriteLine(video.title + " on " video.Player.title);

This is still a very early alpha - I haven’t even checked in the unit tests yet, so the API will very likely change and may not even work (I haven’t been able to test it using a remote connection as my current network setup doesn’t allow my to open the port). Please try it out and send me feedback or pull requests.

I will be dogfooding this API - I’m working on an app that uses it for Moo Moo

Why can't all laptops use the same charger?

Fri, 01 Aug 2014 06:29:53 +0000

As the title really, why can’t all laptops use the same charger - including laptops from the same company.

I’ve trvelled 45 minutes to the co-working space I’m working at only to find I’ve forgotten my MacBook Pro charger, and it’s only got 10% left. I have my Surface and the charger for that, but of course it’s not the same. I can’t even borrow one as the only free chargers here are for MacBook Airs - different plug. This means I’ll be heading back home soon.

Why can’t they all just use the same charger - like with phones where the EU has pushed for all phones to use mini-usb to charge. If we could have a USB spec that provides enough voltage for the laptop to charge then have them all using USB chargers then life would be so much easier.

Portable class libraries

Wed, 30 Jul 2014 07:16:02 +0000

Portable class libraries (or PCLs) are the flavour of the month at the moment. They are a .net library designed to be cross platform - so they work just as well on Windows 8 as they do on Xamarin. Where Microsoft are supporting Xamarin as much as possible, they are releasing a number of their core libraries as PCL.

Unfortunately, where they are only recently a big thing, NuGet package support is limited. I’m writing a client library for the Plex media server which I want to use from a Xamarin app (and will put up here on GitHub once it’s ready) and this makes hits the Plex REST API for it’s communication. Luckily, REST is easy on .Net thanks to RestSharp, an awesome NuGet package that simplifies creating REST clients.

Problem is, it’s not PCL. There is a PortableREST package but this doesn’t have authentication which I need. In the end, I decided to roll my own using Microsofts PCL HttpClient.

Another downside is the support for different platforms. Creating a PCL requires selecting which platforms will be supported, and not all of them have all the classes you may want to use. For example, I’m using ObservableCollection, which is not supported in Silverlight 5, so my PCL doesn’t support that platform. Annoyingly, when you change the supported platform, you have to re-install any NuGet packages that target multiple platforms to get the version that targets the platforms you’ve selected. Seems a bit of a faff really.

Going forward, I plan to make all my libraries PCL. Hopefully more and more people will also do this. And hopefully, the process will get easier.

Favicons

Tue, 29 Jul 2014 04:21:04 +0000

After changing my avatars, I decided to update my Ghost blog to show my new avatar as the icon in the browser/bookmarks and springboard icon on iPhones/iPads.

Coming at this with limited HTML/Web experience I had no idea where to start, so thought I’d write this as a guid to help anyone else wanting to do the same thing.

What you need is called a Favicon. By default the web browser picks up a file called favicon.ico from the web site root, but it can be changed by using the link tag.

<link rel="icon" ... />

Wikipedia has a good article on it here. Any normal image type can be used assuming you want it to work in the proper browsers (Chrome/Firefox/Safari/Opera), but for IE prior to version 11 you have to use an .ico file. Converting to ico’s is easy - there are plenty of sites that will do it for free. I used IconConverter for mine.

This works great for browsers, but Apple has the ability to use icons when adding a shortcut to your site to the springboard. It supports multiple sizes to match the resolutions of different devices (so watch out when a new device comes out with a different resolution, make sure to update the icons). The details of this are here, but it essentially boils down to one link per icon size:

<link rel="apple-touch-icon" href="touch-icon-iphone.png"/>
<link rel="apple-touch-icon" sizes="76x76" href="touch-icon-ipad.png"/>
<link rel="apple-touch-icon" sizes="120x120" href="touch-icon-iphone-retina.png"/>
<link rel="apple-touch-icon" sizes="152x152" href="touch-icon-ipad-retina.png"/>

The names in the hrefs are to indicate which device uses which icons, but you can use your own names of course. The icons get the normal Apple reflective shine etc added to them - but this can be stopped by adding -precomposed to the the rel attribute:

<link rel="apple-touch-icon-precomposed" href="touch-icon-iphone.png" />

The icon sizes must match the required size for the device. These sizes are:

iPhone - 60x60 (note iOS6 was 57x57, iOS7 is 60x60)
iPad - 76x76
Retina iPhone (iPhone 4 and later) - 120x120
Retina iPad (third generation and later) - 152x152

Note that the original iPhone size was changed in iOS7 - so whenever a new iOS version comes out it’s worth checking if there are any size changes.

To get these working on my Ghost blog, I edited the theme. I am using a modified version of Ghostion which I will be tidying up and making available publicly when I get a moment. One note about Ghost - it provides it’s own favicon in the root, so just by adding one to the root of your theme won’t make it work - you will still get thge Ghost one. You have to place it somewhere other than the root and add links to the icons in your header.

Note I made a few changes to this after a comment from RealFaviconGenerator. It looks like they have an awesome site to generate all the favicons you could need automatically. I haven’t tried it out myself yet but will do next time I need one.

Social networking and how the world sees me.

Tue, 29 Jul 2014 03:48:22 +0000

When starting to develop your online brand, there is many things to consider, including how the world sees you. The first thing anyone looking at your public profiles will see is your avatar - the image you use to define who you are. As there are so many different social networks or areas with profiles, it’s easy to have fragmentation in your public avatar. A quick scan of my profiles has lead to quite a few:

Ghost
Twitter
Facebook
Github
Gravatar
LinkedIn
Google+ (does anyone still use this? I just post to it to help google index my posts)

Across these I had 4 different avatars - all of which are probably not good for a public brand. Two have me with my daughter - I love her to bits but I’m representing me publicly, and don’t want my family to be involved in my brand and want to protect their privacy. The other 2 have beer in my hand, including a nice one of me a bit drunk at the Hong Kong Rugby Sevens. There is nothing wrong with beer, and nothing wrong with getting drunk (responsibly of course) but again with a public profile it’s good to avoid anything that could predjudice people against me - it’s easy to look at one photo and make assumptions about the people in it.

My new avatar was cropped from a photo taken of me recently - I did a scan of all my photos to find something suitable. Not overly boring and smart - this does represent me as a person, not me in a suit being formally interviewed - but shows a bit of personality.

Taking a break

Wed, 09 Jul 2014 10:26:28 +0000

I’m taking some time out - just a couple of weeks. Spending time with my family in the UK before moving to Thailand to start my adventure in the world of startups and journeyman programming.

Will be back and blogging again soon.

Setting up a website on Heroku

Thu, 26 Jun 2014 00:32:41 +0000

Now that I’m (almost) out of the rat race and working on my own apps, I though it was time to create a website to host my work. I haven’t got a company set up yet (that’s coming soon) but I’ve been watching a rather excellent Pluralsight training course on the MEAN stack. In this course they use Heroku to host a Node.js application, so I though I’d try it myself.

The advantages of Heroku that I can see are:

Price - for 1 dyno, enough for a small load there’s nothing to pay, no need to even give a credit card. All free. Great for development or a small site. Scaling up is easy as well, just hand over the credit card and increase the number of dynos.
Ease of setup - once you have an account, you just install the developer toolbelt, link your git repo to a remote Heroku repo and push the code. You can push to an app you create on the Heroku site, or it’ll even create the app for you.
Simple configuration - getting things like a domain name set up is trivial.
Loads of add ons - admitedly you have to hand over a credit card for verification first, but there are loads of easily available add ons like MongoDB, analytics, email, the list goes on. All seem to have free tiers as well, again ideal for development.

I started with a basic node express app, nothing too fancy, just a ‘coming soon’ page using bootstrap. Once this was running locally, I uploaded it to my private GitHub repo. This was my starting point. The total time to get it running in Heroku was about an hour, and this is coming from never having done it before - and half of that was trying to figure out why it wasn’t picking up the port from the environment variables (turned out I pushed the wrong branch). I won’t go into the details of how to do it, the Heroku documentation is excellent and the Pluralsight course mentioned earlier goes through the details as well. DRY should apply to blogs as well you know!

Overall I was very impressed at how simple and easy it was. I think I’ll continue using Heroku for my website, especially when I add more features such as MongoDB. Seeing as it runs Node.js as well, I should be able to create the servers for the apps I’m planning to write on it with ease.

Check out my website so far - not much but more is coming soon.

MooMoo.io

Waiting again

Wed, 25 Jun 2014 21:55:40 +0000

As I’ve mentioned before, I’m taking 6-9 months out of the rat race so I can do my own thing, including learning some new skills to take into the world of work. The skills in question are HTML/Javascript and Xamarin.

Javascript is becoming ubiquitous, it’s the assembly language of the web and for any startup being able to create a web site or web application is important. I’m going to learn it alongside Node, HTML and all the tooling in that space. It’s hard actually as there is so many frameworks, so many tools to learn so I’m going to focus on the MEAN stack. There is an excellent Pluralsight course on this - worth a watch if you have a subscription (what do you mean you don’t - get one already). I’ll be using this to create my web site - more information coping soon.

Xamarin is breaking ground in cross platform application development. Not content with allowing C# apps to be run on iOS and Android they went on to introduce Xamarin.Forms to allow cross platform UI development using XAML. I’ve been developing in C# for the past 7 years so it’s the natural next step for developing in the mobile space. Having done desktop applications for years in WPF it make sense to expand my skills to include mobile.

Why is this post entitled waiting? I’m waiting for my Xamarin license. They’ve really helpfully put together a subscription that matches both my needs and my meagre unemployed person budget and I’m just waiting for the invoice. It’s been a few days now and although I appreciate they are busy and these things take time, I’m itching to get started! My own impatience is getting the better of me. I know I can develop using the 30 day trial in Xamarin studio but I’m licensing a version that includes Xamarin for Visual Studio and I really can’t wait to produce an app in VS for my iPhone.

Synergy

Mon, 16 Jun 2014 11:52:09 +0000

As you’ve probably seen from my other posts, I’m developing iOS apps using Visual Studio on Windows as my development environment, and a MacBook Pro for running the actual builds. This is a pain when I want to test. As good as Ripple is, it’s not as good as the actual iOS simulator. It’s a pain jumping from one computer to the other.

Thats where Synergy comes in. It’s a small app you install on all your computers and it allows you to use your mouse/keyboard on them all. So for my setup I have my surface as my left most screen, a 24" cinema display as my center screen connected to the Surface, and my MacBook Pro as my right hand screen. I move my mouse across to the right and it appears on the Mac, so I can use all the Mac apps without changin keyboard or mouse. Very useful. It’s free as well, but they do ask for donations, so you can try it out and once you realise just how awesome it is you can donate as much as you think it’s worth (they suggest $10).

Popping my open source cherry

Tue, 10 Jun 2014 22:06:57 +0000

Today the pull request for my first ever contribution to someone else’s open source project was accepted and merged.

It was a small bug fix to Framework7 - a really cool HTML5 framework for building iOS 7 style apps. I’m using it for my first cordova project and what it outputs looks just like a native iOS 7 app. Highly recommended.

A big thank you to Vladimir Kharlampidi for creating this project.

Even though my fix was small, I did feel a bit nervous once I commited it. Would it be accepted or laughed at. It was quite a rush when I got the email saying it had been merged.

One done, many more to come.

Lets eat!

Mon, 02 Jun 2014 23:11:41 +0000

So once again I’m waiting, this time whilst I download the XCode 6 preview. I’m working on a simple node application for photo management - uploading, storing, showing thumbnail previews and showing the pictures. Nothing complicated but more of my self learning of node and javascript. Seeing as Apple have just announced Swift I thought I might try to develop a simple iOS app in Swift that talks to my node app. Once XCode is dowloaded over my pathetic broadband (5 hours 28 minutes remaining) I’ll give it a go and blog about it.

In the meantime, I decided to eat some dessert. I make a mean version of banoffie pie, using raspberries which add a tartness that cuts through the sweetness of the pie.

The recipe is simple. What, a recipe on a technology blog? Well why not - developers have to eat after a hard day pumping mouse and keyboard.

Base - melt a couple of ounces of butter in a pan, and add a small pack of finely crushed digestive biscuits (bung them in a ziplock bag and roll over them with a rolling pin). Press into a greased pie dish to make a bottom and sides and slap in the fridge.

Toffee - put an unopened can of sweetened condensed milk into a large pan of boiling water and boil for 3 hours. Make sure the can is always completely covered with water. After 3 hours, open the can and spread the contents into the pie dish. Be warned, the contents of the can is stupidly hot, will spurt out when you pierce the can with a can opener and sticks like shit to a blanket. Best to open it wearing oven gloves. Bung the pie dish back in the fridge.

Once the toffee is cooled, cover in raspberries. Then whip some cream with some hot chocolate powder (not cocoa, it’s too bitter, needs to be a decent hot chocolate powder) and spread on top.

Then eat. Nom, nom, nom.

Using Visual Studio to develop Cordova apps part 2 - Creating an app

Tue, 27 May 2014 00:10:02 +0000

Building on the previous post in this series, I thought it would be fun to try to learn Angular.js and include this in my Cordova app. I could also do with so nice looking widgets, so I decided to have a play with TopCoat.

First thing I need is an app to build. One of my current ‘First World Problems’ is having to convert from US units (mg/dL) to international blood sugar units (mmol/L). I’m a diabetic and I’ve been living most of my life in the UK and in Hong Kong - two countries that use international units for blood sugar measurement. I’m now living in Bermuda, and they only sell US spec blood sugar meters, and provide testing strips that only work with these small set of meters. The meters are factory locked to US units for some reason (unlike international meters that can be changed). So for now, I’m stuck with testing in US units.

The conversion is 18 to 1 - so a US measurement of 90 is the same as an international measurement of 5. Simple enough, but I’ve got better things to do than learn my 18 times table, like create an app to do the conversion for me.

So, lets create an app.

Building the app

I’m starting with the simple HelloWorld example from the previous post, and changing it to suit my needs. The UI is really simple - a top bar and two input boxes.

The top bar is the TopCoat navigation bar

<div class="topcoat-navigation-bar">
 <div class="topcoat-navigation-bar__item center full">
 <h1 class="topcoat-navigation-bar__title">Blood Sugar Converter</h1>
 </div>
</div>

This gives a nice header bar. I’m using the TopCoat mobile dark skin, I really like the way it looks.

For the input boxes I wanted to select all when you press them - that way you can overtype instead of having to delete the existing contents. A quick search of Stack Overflow gives a few suggestions to select everything, but these lead to a problem. When the text is selected the iPhone pops up a menu with standard options for selected text, Cut, Copy etc. This is far from ideal as it looks really bad. In the end, I just set the value to undefined on the click event handler. This clears the box so there is nothing to select allowing you to type the number without having to delete everything. The fun part with the input boxes was the conversion. Rather than having to click a button to convert, I wanted it to happen on the fly. This is where Angular comes in. Angular allows you to bind the values from input boxes to variables - you update the input, the variable automatically updates, you update a variable, the field showing it automatically updates. It also provides events to detect value changes.

So first off, a controller.

// Create the angular module
var bloodSugarConversionApp = angular.module('BloodSugarConversionApp', []);

// Create the controller
bloodSugarConversionApp.controller('BloodSugarConversionCtrl', function ($scope)
{
 // defines the initial values for our variables
 $scope.usMeasurement = 0;
 $scope.internationalMeasurement = 0;

 // this function is wired up to a change handler for the US input field
 // when this changes, it updates the international measurement
 $scope.changeUSMeasurement = function()
 {
 // setting the international measurement updates the value in the input
 $scope.internationalMeasurement = Math.round(($scope.usMeasurement / 18) * 10) / 10;
 };

 // this function is wired up to a change handler for the international input field
 // when this changes, it updates the US measurement
 $scope.changeInternationalMeasurement = function()
 {
 // setting the US measurement updates the value in the input
 $scope.usMeasurement = Math.round(($scope.internationalMeasurement * 18) * 10) / 10;
 };
});

So, what’s going on here. First we define a callback that creates the BloodSugarConversionCtrl controller. This function sets up the $scope to have a couple of variables for the two units of measurement initialised to 0, and declares a couple of functions on it that will convert when a value changes. This controller is then wired up in our HTML.

<div ng-controller="BloodSugarConversionCtrl">
 <p class="label-text centered">mg/dL</p>
 <input ng-model="usMeasurement" ng-change="changeUSMeasurement()"
 onclick="this.value = undefined"
 type="number" step="0.1" class="topcoat-text-input centered" />
 <p class="label-text centered">mmol/L</p>
 <input ng-model="internationalMeasurement"
 ng-change="changeInternationalMeasurement()"
 onclick="this.value = undefined"
 type="number" step="0.1" class="topcoat-text-input centered" />
</div>

At the outer <div> we set the controller using the ng-controller attribute, and this is accessible to the inner elements. We then set the model using the ng-model attribute against the inputs - this binds the given variable to the value for the input. Finally we wire up the events - the onclick to clear the value when selected (as described above), and the ng-change to handle the value change and update the corresponding value.

And that is basically it. Angular takes care of the hard parts.

When I ran it, it did look a bit bad. The app could be scrolled - it’s essentially a hosted web page so the scrolling is a default behavior. It also scrolled up when you selected the input boxes so that the keyboard could fit on the screen, and this made the status bar visible through the HTML elements. Scrolling was easy to turn off - just add a line to the config.xml

<preference name="DisallowOverscroll" value="true" />

Removing the status bar was harder. There are many solutions on the web for earlier versions of Cordova or earlier versions of iOS, but for Cordova 3.3.1 and iOS 7 the only way I could get it to work was using the StatusBar plug-in. Install it by adding an entry to the vs features in the config.xml

<vs:feature>org.apache.cordova.statusbar</vs:feature>

Then hide it in the onDeviceReady event

StatusBar.hide();

Done.

Deploying to the App store

Once done, I thought it only fair to share the app with the world for free on the app store.

Doing this was fairly simple. The Cordova build engine that runs on my Mac spits out an XCode project for the app for every build. It’s a simple job to load this into XCode, set the relevant provisioning profiles and upload it to the store. The only gotcha is the version number. The default one in the config.xml is 1.0.0.0. iTunes app submissions only allow a 3 part version, so you’ll have to update this to 1.0.0 or similar before it will pass validation.

My app is currently awaiting review - I’ll update this post with a link once it’s passed and available.

The source for this app is available on GitHub. I’m new to HTML and JavaScript so please be kind. All comments and criticisms greatfully received.

What to do when you're stuck inside.

Mon, 26 May 2014 13:04:22 +0000

It’s a glorious day here, a perfect day to take my wife and daughter to the beach. Unfortunately we currently live in Bermuda, an island paradise with stunning beaches and clean water. Why is this unfortunate? Because the country is so badly run. Today is Bermuda day, and they are celebrating by closing one of the main roads for a half marathon. This road is the one that my house is on. So for the rest of the day, I can’t get the car out and drive to the beach. I can’t walk either as I’ll have to walk on roads with no pavement that are open that are narrow and the drivers here are idiots, so it’s not safe with a child. They didn’t tell us the road would be closed, no sign on the exit from our estate - my wife popped out with our daughter and got shouted at by people on the side of the road and the police and made to leave her car in the local supermarket and walk back. Not her fault - there was nothing to indicate that the road would be closed. But thats the way here - hence why we are leaving soon.

So I thought today would be a good day to tidy up a few things in the running.

Tidying Up

Firstly I finally tidied up my desk so I could work using a proper keyboard and monitor. Most of my recent dabbling was using my Surface on a coffee table in front of the sofa whilst my wife was glued to ‘Americas Next Greys New Girl Anatomy’ or whatever it is she watches. My setup isn’t the latest and greatest, but it does what I need. My Windows development box is a Surface Pro 2, the 256GB model with 8GB RAM. It’s a great Windows 8 tablet (shame Windows 8 isn’t a great tablet OS - not having a ‘Modern UI’ browers unless IE is your default being the worst part), and a pretty powerful dev box. I’ve got the type keyboard which is fine to type with but the trackpad is rubbish, not a patch on the Apple track pads. For input on my desk I prefer a mechanical keyboard and decent mouse. Logitech do some pretty decent kit so I’ve got myself a G710+ keyboard and Performance MX mouse - a darkfield one as my desk has a glass top. The surface has a nice screen which works beautifully as a small second, with an old 24" Apple Cinema display as my main monitor. Add a 13" MacBook Pro for OSX/iOS development and I’m all set. My desk as mentioned before is a glass topped one. Nothing great, just a cheap Ikea number. I don’t have a great chair (though I’d love an Aeron) but instead I use a large excercise ball. Work those abs whilst working the fingers. Add a Kinect and a load of other junk and away I go.

Blogging

Secondly - this blog. I wanted a better theme so I’ve installed Ghostion, a really nice looking theme based on Foundation 5. I had to make a few tweaks to make it look better on my iPhone which was a good excuse to get my hands dirty with Foundation. I’m still very new to HTML, Javascript and all the eleventy billion frameworks out there so it took a while, but I’m pleased with the result.

I really like the way with Foundation that you can hide elements based on the screen size with a simple css class.

<div class="row hide-for-small-only">
 <!-- Large screen content -->
</div>

<div class="row show-for-small-only">
 <!-- Small screen content -->
</div>

Ghostion has some really cool features that work out of the box with minimal configuration. Disqus for commenting or Google Analytics are all available by just adding the relevant id’s to one file.

Coding

Finally, time to do some coding. The rest of today will be spent on a couple of apps. Years ago I did a game in the iOS app store, which I’m planning on resurrecting and re-releasing and putting the code on GitHub. I also want to finish playing with Cordova apps using Visual Studio, so I’ll try to get something from there on the app store. Blog posts on these coming soon!

Trust

Tue, 13 May 2014 23:52:13 +0000

I wrote this post on Medium a while ago, so as part of my attempts to bring together all my work into as smaller number of places as possible, here it is.

Shouldn’t our default position be to trust first?

Trust is good. My advice to my friend — go somewhere where you are trusted.

Using Visual Studio to develop Cordova apps

Tue, 13 May 2014 23:35:37 +0000

So after yesterdays attempt to create a ‘Hello World’ app using the new Cordova tools for Visual Studio, I thought tonight I’d try again.

If you haven’t heard the news, Microsoft announced a CTP of support for Cordova inside Visual Studio. Thats right, Microsoft is opening up to a world of other devices. The aim is to allow you to build HTML5 apps inside Visual Studio and deploy to Windows Phone, Android and iOS, all from one place (with a networked Mac for iOS building). I guess this might put Adobe’s PhoneGap out of business, but hey, thats how it works when you build a business exposing open source software.

I’m writing this as I go along trying to do a ‘Hello World’ app, and so far not much success.

I’m working through the documentation and here are my experiences:

First I did the basics - I launched Visual Studio and created a new solution using the new Multi Device Hybrid App template. I set up my MacBook Pro to host the remote build and simulation agent, set the build output to be the iOS simulator and hit build. First problem - it couldn’t find Java. It seems the installer didn’t successfully install the Java 7 JDK. One more delay whilst I download and install this.

Once the JDK was installed - I tried building and running again. Next thing I know, up pops Chrome (with the obligatory set your default browser prompt - I use IE as my default simply so I can use a bowser on my surface in Metro mode, oh for Firefox for Metro) running the iOS simulator with my ‘Hello World’ app running. WooHoo!

Now to try the first interesting bit - can I run the app on an iOS simulator on my Mac? Lets set the output on Visual Studio and see what happens! So far so good - things are happening on the Mac’s terminal. And as if by magic:

I’m in shock. No third party paid for components like Xamarin, just an installer from Microsoft that installs open source code and I have an iOS simulator on my Mac running code from Visual Studio. This is groundbreaking stuff (to a techy of course - it didn’t cause my wife to even look up from whatever celebrity gossip site she’s browsing).

Lets take it all the way - lets see if it can run on my iPhone itself.

Seems you can’t have it all - Code signing error. I guess I need to provision my iPhone. BRB!

Time to let a picture paint a thousand words:

Excuse the poor quality shot - it’s late and I couldn’t be bothered to turn the lights up. Thats the same app - the basic Visual Studio Multi Device Hybrid app template built on my Surface using my MacBookPro as a remote build agent, installed using iTunes on the Surface running on my iPhone.

iPhone development has just got a lot easier.
Tomorrow I’ll try adding some code and developing a simple app.

Waiting

Mon, 12 May 2014 15:36:15 +0000

Every journey begins with a first step. Unfortunately that first step is not always made as soon as the journey is ready to begin.

Over the past few days I’ve been starting the preparation for a new life. My family and I are escaping the rat race for a while. I’m working out the notice period on my nine-to-five, my wife is packing up the house and we’re taking a break from the normal life to allow me to focus on growing my tech career - a chance to contribute to open source or blog without the IP clause in my job contract limiting what I can post, a chance to try out a few app ideas and work towards a startup or two (and probably fail along the way), and most of all a chance to not work the norms of a five day week and allow me to work when I am at my best, and fit time in with my family when it is best for them.

Getting Started

As well as mental preparation, at some point you have to break ground. Today I purchased my own domain name, signed up to Ghost to start this blog and decidec to write my first technical post. Today Microsoft announced support for Cordova in Visual Studio. Seeing as I have a few startup ideas that will involve connecting the desktop to mobile, it piqued my interest.

So this evening (after the latest Game of Thrones of course) I decided to do a basic Hello World developed on my Surface and running on an iOS simulator.

This would be my first step on the journey.

But, like all first steps there is the preparation. You can’t take a walk without putting on your shoes. And tonight my laces are taking a long time to tie up. All down to Internet speed. At the moment I am waiting for the tooling to install. It has to do a large download over my slow internet connection, then install everything. All this before I can begin. At this rate I’ll be tucked up in bed with my enthusiasm drowned in bourbon before I’m even asked for the obligatory re-boot.

Currently it’s the Android SDK that is taking the time - which is doubly annoying as I only want to develop for iOS.

Ah well, at least I have time to set up my domain to point to my Ghost hosting and write this post.