September 30th, 2015
By Dr. Tim Saxe, Ph.D, QuickLogic Corporation
Not to mix metaphors, but when the “swarm” is a distributed network of sensors checking multiple personal health and fitness indicators, we could be opening a can of worms.
“Here Comes the Swarm” isn’t a B movie, it’s a prediction. Yes, the swarm really is coming. But this swarm won’t be teeming with killer bees or ravenous locusts or harbingers of the zombie apocalypse. Instead it’s made up of a multitude of sensors—covering our bodies and monitoring a wide range of our daily activities and various bodily systems to promote our wellbeing. And the fact that we are rapidly moving from today’s reality of a handful of sensors integrated into a single platform such as a smartphone, to tomorrow’s vision of dozens or perhaps even hundreds of sensors placed across a person from head to toe, engenders a host of implications that we as an engineering community are only starting to contemplate.
Figure 1: Which device will have the authority to evaluate and prioritize data from multiple sensors?
Head and Shoulders (and Knees and Feet and…)
Unfortunately, there is no single ideal place for wellbeing sensors. Some sensors need access to the skin to measure things like heart rate, while others need access to the air, to measure things such as air quality. Some work better on the feet and others on the waist, chest, or head. Therefore we can assume that, as sensor costs come down, we will start to see sensors distributed across our bodies—each placed in their respective near-optimal locations. Some examples already exist, for instance the Garmin Foot Pods and the Mio Velo and Link heart rate monitors.
As sensors become more widely distributed, it will ultimately make sense for them to be wirelessly networked together, probably using a communications protocol such as the low-energy Bluetooth Smart standard. Once that happens, we will find cases of duplicated capability, which will create the opportunity to select among sensors based on their location and resulting quality of data. For example, a wrist is a difficult place to get accurate step counts because arms move even when people are not walking. A smartphone in a pocket does not have that problem, but relying on the smartphone might risk missing data when the user is walking and the phone is not on the user, perhaps being charged.
Accuracy and Power Considerations
If we think of the smartphone and a wrist wearable as part of a personal sensor swarm, then swarm fusion can determine which device, wrist or smartphone, is giving the more accurate data and choose accordingly. Similarly the ear is a much better place to monitor heartrate, so when a person is using HRM enabled earbuds, the swarm can use data from them rather than from a wrist mounted device.
Another consideration is power. With an intelligent swarm of sensors, the swarm can decide which sensors are not required at any given time and power them down. Again, for example if a person is walking with both a wrist device and smartphone, since the smartphone has a larger battery and superior quality data, the swarm can shut down the pedometer function in the wrist device and rely on the phone, at least until the user removes it from their pocket.
Having a distributed network of sensors creates other issues. For example, which device (or set of devices) makes the decisions about data quality or power or whether or not a particular sensor is functioning properly? A likely starting point would be for one device, perhaps the user’s smartphone, to be the “main” device and to have the authority to evaluate and prioritize data from multiple sensors. These might include sensors integrated into the smartphone itself, as well as other sensors distributed across the body. For example, in the simplest case a phone might look for a foot pod, and if it finds one in the swarm network it could turn off its own pedometer to save power.
Decision Distribution
Eventually the decisions will be distributed, at least partially. Devices will broadcast their capability, and, based on the capability of those around them, will either shut down or continue sensing and transmitting data. Extending our pedometer example, a foot pod might broadcast that it is currently counting steps (with high accuracy) in which case both the user’s smartwatch and smartphone would halt their step counting function. If the foot pod stopped broadcasting, then both the watch and phone would start. Then, based on that scenario, the watch (knowing that its step counting function is less accurate than the phone’s) might halt its pedometer.
Accelerating the Process
Managing either centralized or distributed decision-making and control will initially be challenging for the industry. If the development of networked personal sensors rolls out in a similar way to past transitions of this nature, initially the results will be painful and frustrating. The first stage will be closed systems from specific manufacturers in which all of their devices communicate with each other but not with devices from other manufacturers. That approach will help manufacturer margins and improve the user experience within the closed system, but will also engender user frustration over their inability to include devices outside of the system. The second stage will be an open network, which tends to commoditize the hardware, but will ultimately work best for users who now have access to everything and can choose best-in-class or best-value, depending on their needs.
One hopeful trend that could really accelerate this process is the role ARM is playing. We all know of ARM as the company that develops the processors that power the mobile world—from the high performance processors in smartphones to the low power M0+ that powers Bluetooth beacons. But ARM is also pulling together the companion IP, software and eco-system that the distributed world of IoT and wearable swarms will need.
Prepare for the Swarm
Elements of the swarm are already starting to appear (Figure 1). There are heart rate chest straps, optical heart rate wrist bands, and foot pods that feed to a smartphone. There are also speed and cadence sensors on bicycles and we can expect to see more sensors showing up on various forms of exercise and sporting equipment. The devices in these systems don’t yet have a network actively choosing data from different sources, or managing power consumption across the system, but it seems likely that soon they will. More and more devices will be added, with increasing levels of wireless connectivity and the resulting opportunities to manage device functionality, data quality and battery life.
So we all should prepare for the swarm. It will be here sooner than we think.Timothy Saxe (Ph.D) joined QuickLogic in May 2001 and has served as Sr.Vice President and Chief Technology Officer since November 2008. Prior to this role, Dr. Saxe served as the company’s Chief Technology Officer and Sr. Vice President, Engineering from August 2006 to November 2008 and as Vice President, Software Engineering from May 2001 to August 2006. From November 2000 to February 2001, Dr. Saxe was Vice President of FLASH Engineering at Actel Corporation, a semiconductor manufacturing company. He holds a B.S.E.E. degree from North Carolina State University, and an M.S.E.E. degree and a Ph.D. in electrical engineering from Stanford University.
I will pull up a few items for serious students to look at in this regard...
But ARM is also pulling together the companion IP, software and eco-system that the distributed world of IoT and wearable swarms will need.
I really enjoyed this one...
ARM’s CEO reveals his IoT wishlist
by: GARY SIMSJULY 9, 2015
629
57110462
There is a 99 percent chance that your smartphone is using a processor based on a design from ARM. Qualcomm, Samsung, MediaTek, even Apple all make processors based on ARM’s designs and/or compatible with the ARM Architecture. But ARM isn’t just into complex CPUs and GPUs. It also designs a range of microcontrollers, and they are proving to be just as popular. In 2014 alone, ARM’s partners shipped some 4.4 billion microcontrollers based on ARM’s Cortex-M designs.
Microcontrollers are used for the jobs which don’t require a full microprocessor. They don’t have GPU, they only have a few K of memory and they run at much lower speeds. As a result they consume much less power. As well as being used to control the display on your oven or as the brains in wearables like the FitBit, ARM’s microcontrollers are also perfect for theInternet of Things.
The first thing he would like to see is new non-volatile memories. “Flash is great, but for IoT devices it’s not so good — it’s power hungry, its area is too big and it’s unreliable,” said Segars. “We need high density with reliability and low-voltage operations that involves not just scaling today’s flash down, but a really low-cost process with fewer steps to integrate into devices.”
A jet engine produces 20 terabytes of data per hour during flight.
Another area where Segars would like to see new innovation is in “Fog Computing.” You have probaly heard of Cloud Computing and if you use any of Google’s services you will be using it (even if you don’t know what it is called). With Cloud Computing all the services are provided via servers and storage somewhere on the Internet. It is cloud in that it is amorphous.
However there is an IoT data onslaught coming and the volumes of data that will be produced by billions of IoT enabled devices is too large to be handled by the cloud. As a result some of the bandwidth, storage and processing of these data needs to happen closer to the actual data source. For example there are some 46 million smart meters in the U.S. at the moment and they are producing 5 terabytes of data per day. Even worse, a jet engine produces 20 terabytes of data per hour during flight. The combined data produced by homes, office, factories, and machinery will far exceed what the abilities of the Cloud.
So the next paradigm is Fogging where some of the bandwidth, storage and a processing is handled locally. This job won’t be handled by a single server, however it will be done piecemeal by the IoT devices themselves and/or by the supporting gateways and hubs. Once the data has been processed locally, the important data will be sent further up into the cloud.
Consider that it will be the same for the mobile device...have to be impressed with Sensory Inc., who want deeply embedded audio recognition. It functions locally so that no cloud is needed to listen. QUIK's implementation of the Sensory algo is better than I could have hoped for.
“This is what5G networks are all about — network providers are looking at the equivalent of set-top boxes in the neighborhood,” said Segars.
We’re at one of the most interesting points in the history of computing.
He also mentioned that IoT is pushing the boundaries on component density. The industry needs to develop new packaging technologies focused on enabling a new class of highly integrated, small, low-cost components.
In closing Segars said, “We’re at one of the most interesting points in the history of computing – it’s been about high performance, but it will become much more about distributed resources.”
This last comment is really key to understanding IoT. Before it was all about performance, the fastest CPU, the fastest GPU, and so on. And that will always be true to a point, however the real power of IoT is in getting lots of low energy devices to work together to produce something that is bigger than the sum of all its parts.
It does plug right into what Dr. Saxe just spoke of.As I have read about the IoT, and a potential role for QUIK in it. I have listened carefully on security...
ARM Expands IoT Security Capability with Acquisition of Sansa Security
30 July 2015
Acquisition will offer hardware and software-based security features, boosting protection for sensitive data and content on any connected device
ARM, Cambridge, U.K, July 30, 2015 - ARM has acquired Israel-based Sansa Security, a provider of hardware security IP and software for advanced system-on-chip components deployed in Internet of Things (IoT) and mobile devices. The company currently enables security in more than 150 million products a year and Sansa Security technology is deployed across a range of smart connected devices and enterprise systems. The deal complements the ARM security portfolio, including ARM® TrustZone® technology and SecurCore® processor IP. Terms have not been disclosed.
"Any connected device could be a target for a malicious attack so we must embed security at every potential attack point," said Mike Muller, CTO, ARM. "Protection against hackers works best when it is multi-layered, so we are extending our security technology capability into hardware subsystems and trusted software. This means our partners will be able to license a comprehensive security suite from a single source."
Sansa Security technology makes it easier for manufacturers to build secure products by offering a complete hardware subsystem that adds additional isolation of security operations from the main application processor. This is complemented by software components operating on top of trusted execution environments to perform security-sensitive operations. The acquisition builds upon ARM's embedded TrustZone technology, creating extra protection against malware and malicious software. It is a system-wide approach that underpins security-related chipset and trusted software needs. This enables the protection of any connected device and management of sensitive data and content.
"Our technology is already being used to protect data gathered and transmitted by a multitude of IoT and mobile devices," said Coby Sella, CEO, Sansa Security. "Joining ARM will enable us to scale the business by helping ARM's global technology partners to address their most pressing security needs. Aligning what we do with the world's leading IP company, allows us to develop our products and capability to new levels."
It is one important area and does QUIK know anything about this topic?
Have they said anything bout it?
Yes, they had a phrase in there... it was what can you use the potential 2nd FFE for and one of the choices is security.
Why would they do that............good security is compute intense and done in hardware.
![[IMG]](http://eecatalog.com/sensors/files/2015/09/sensors_image.jpg)
![[IMG]](http://eecatalog.com/sensors/files/2015/09/exec-tim-saxe.jpg)
![[IMG]](https://lh3.googleusercontent.com/blogger_img_proxy/AEn0k_vUdzTaKaIczw-QIcnmTLUY8V53wfMtIcgZ-pLJ77_72MYOJnZLIbDExky2A-qThwwhpo3mdfXL9OL4mHVAmrN9YuW2_Hr6J50XiiQocHr-NJGjJoJ7sBWwZc6mjAHifIms8j6OmFmhAIwCda8igCwJoLP2P9BDNZwwsLPH6nkp1pRdnjA=s0-d)
![[IMG]](https://lh3.googleusercontent.com/blogger_img_proxy/AEn0k_tE3PhXYD6UUiamXq13LhK4IQjBHw2T7nYK0t0JBtynlzVWXbTNZTmTH6fMs5CFMKnEG-QQfMIghbH-aAZxdaRqKxwCfm0fPo9e6Ec7qmEBi39UuTZ5lh13bDMxtL94SgUnh4WC4_cX80Tyh776tWUaBng3BZWfqEfuOkY_xWc7MpUu=s0-d)
commentary.
As the adjacent possible that QUIK has been exploring becomes more visible consider the diversification that you get now.
So far from a one trick pony whose idea needs to get adopted.
Fitness
Screenless audio UI
Compute intensity of Indoor location
it starts to be something very hard to find.
NOKIA FORGES ALLIANCE WITH SAMSUNG, SONY TO REVOLUTIONIZE INDOOR POSITIONING
By Francis Bea — August 25, 2012
GPS technology isn’t accurate to a pin point, at least not yet, and definitely not indoors. To determine an exact location within an enclosed space requires external hardware for the purpose of mapping an indoor location, and it’s a troublesome task that many companies, like Wifarer, are diligently working on. To help phones better breach the indoors, Nokia has formed an all alliance of 22 different companies to cooperate to develop accurate indoor positioning technologies for implementation into mobile devices.
The alliance, called In-Location Alliance, will develop and promote short-range wireless technology for the purpose of indoor positioning to a minimal degree of inaccuracy using mobile devices. For example, a mobile app for the purpose of aiding shoppers could offer a shopper browsing the cereal aisle of a grocery store coupons based on the specific location of the shopper. Other scenarios may allow the technology to help with finding the one shoe shining booth within a mall as large as The Mall of America.
The alliance is already close to perfecting this technology thanks to Nokia’s head start, which will likely be baked into the alliance’s efforts. In its testing phase, Nokia’s High Accuracy Indoor Positioning Solution (HAIP), which uses Bluetooth 4.0 and minimal power consumption, was able position a mobile device to the accuracy of one meter. Through further development and fine tuning, Nokia will be able to pinpoint a device to within an accuracy of a mere 20 centimeters.
“The key criteria for the indoor positioning technology are high accuracy, low power consumption, mobility, and the low cost. The solution has to be easy to implement and easy to use,” Nokia’s Boc Ly wrote in a blog post.
Nokia by itself has already developed the technology for indoor positioning, which raises the question about why the company should even work with other corporations at all. Nokia explains that due to the technicalities, developing the technology for indoor positioning requires different roles including telecom operators, system manufacturers, application developers and handset manufacturers.
the phrase "ecosystem partners" resonates in this endeavor
For the technology to work, there are multiple facets to the equation. First, there are two hardware components to the technology. A HAIP antennae must be installed in the ceiling or the area from the reception of the wireless signal, which will map out the indoor location. The second component can be installed in a mobile device or as Ly explains, the component “made as separate tags, which can be attached to any asset.” Then the more daunting effort, and where the alliance will really come into play, is to convince every original equipment manufacturer and building owner to install these components in their effort to provide a complete network for pinpoint mapping.
“The alliance members can bring the technology to the market sooner and also extend the technology in the future and provide new use cases and opportunities,” Ly writes.
More specifically the In-Location Alliance, comprised of powerhouses including Sony Mobile Communications, Samsung Electronics and Broadcom, and its role is in three parts:
Competing technologies exist, like Wifarer and Navizon I.T.S. And with demonstrations already set to take place later this year, it may be only a few more years before we’ll be finding the nearest bathroom in an airport using Nokia’s HAIP technology.
- Continue working together on system architecture based on a standard based indoor positioning solution.
- Alliance members will prepare and execute pre-commercial pilots and practical demonstrations starting in the second half of 2012.
- Alliance members will brainstorm and evaluate new use cases and new business opportunities based on indoor positioning technology.
Check out the video of Nokia’s HAIP below:
Read more:http://www.digitaltrends.com/mobile/nokia-samsung-sony-in-location-alliance/#ixzz3mkOWh2N9
Follow us: @digitaltrends on Twitter | digitaltrendsftw on Facebook
![[IMG]](https://s3.amazonaws.com/sr-company-logo-prod/5135d6abe4b0b42dcb45c26c/huge?_1408022311110){kind=logo}
