The importance of data preparation has grown exponentially with the rise of AI. Data comes in many forms and formats, including homegrown applications, SQL databases, files, sensors, video, and physics-driven analog data. Traditionally, data cleansing is defined as detecting and correcting (or removing) corrupt or inaccurate records from a dataset, table, or database. The data challenge presented is identifying the data's incomplete, incorrect, inaccurate, or irrelevant parts and then replacing, modifying, or deleting the dirty or coarse data.

Please make sure to sign up for this exclusive summit in Novi, MI!

Since our company's inception, we have had the privilege of engaging with a diverse range of customers, all grappling with a common challenge-the management of their data. Whether a small business or a large enterprise, the issue of technical debt, a result of short-term thinking about data, is a consistent theme. Technical debt, in this context, refers to the cost that accumulates when short-term solutions are implemented to address immediate needs, leading to a complex, inefficient, and duplicate data infrastructure over time.

This technical debt spawns from a patchwork of applying technologies one by one, reacting to a current need. A prime example is when companies opt for a do-it-yourself plan because they don't have to engage procurement or IT and currently have resources available. Aside from the expanding technical debt, this strategy ignores the ongoing service support issues that cost 10x what an off-the-shelf product when looking at the total cost of ownership.  I have observed extreme examples of this behavior at companies that have been around for many years. The decision to patch things and solve a short-term pain is very tempting. Leaders convince themselves they can do it cheaper and better because it is custom-built for them versus the pain of trying to convince stakeholders to procure a product.

It's clear that addressing and reducing technical debt is not a simple task. It requires a structured methodology for identifying, justifying, and funding new projects. This funding is not just about acquiring technology; it's about generating the emotional momentum needed to overcome the inertia that has built up over many years in replacing outdated technology.

Traditionally, the data cleansing is defined as the process of detecting and correcting (or removing) corrupt or inaccurate records from a record set, table, or database and refers to identifying incomplete, incorrect, inaccurate or irrelevant parts of the data and then replacing, modifying, or deleting the dirty or coarse data.

This is post 2 in the series on 'The 7 Data Habits of Highly Effective Product Companies'. Tune in to our LIVE WEBINAR '7 Data Habits of Successful Product Companies' - MARCH 31, 11 a.m. CDT

My day job for the past 30 years has been helping people with their sensor data. In my time off I tinker with and build PCs, try my hand at some home improvement projects and stay on top of new technology developments through the consumption of copious amounts of online tech news website articles and YouTube videos. What I have learned—and keep being reminded of time and time again—is that there are plenty of things that I don’t know. And, that sometimes I’m not even aware of things that I don’t know. Let me explain.

Car & Driver's March 2019 issue includes an article on sound Active Noise Cancellation (ANC) and Electronic Sound Synthesis (ESS). With the September 2019 deadline looming for auto manufacturers to meet PEDESTRIAN SAFETY ENHANCEMENT ACT OF 2010 (PSEA), which governs EV & HEV sound emissions, there is opportunity for new automotive products that detect and emit sound at low speeds in these vehicles (< 18.6 mph). There is also the opportunity for the reverse, removing annoying sounds. Innovative sound creation and sound reduction systems are relying on sensor data to determine what sounds to synthesize and what sound to deaden.

National Instruments will be hosting their annual customer conference and partner showcase, NIWeek 2019, this year from May 20-23 at the Austin Convention Center. NIWeek draws engineers from across the transportation, aerospace, medical device industries and many others. There are also  a fair number of members of the academic community rounding out the crowd. It is a hub of activity for Viviota customers and partners and a busy time for our team. We were delighted to find out in January that Viviota will be presenting two of the conference technical sessions:

"What we've got here is a failure to communicate." In the 1967 Paul Newman movie classic Cool Hand Luke, that memorable line is uttered by the character "Captain" (head of the chain gang) to justify beating Cool Hand Luke, played by Newman. Captain of Road Prison 36 has just told Luke that he is wearing chains for his good—to which the prisoner cleverly responds, "Wish you'd stop bein' so good to me, Cap'n." 

The automotive industry is working toward a revolutionary event, a truly autonomous vehicle—one in which a human driver is no longer required. To accomplish this, automotive companies have focused R&D teams, whose mission is evolving and growing onboard automated safety systems. Today onboard safety systems include features such as airbags and anti-lock brakes. For autonomous driving to become a reality, more dynamic safety systems are needed, for example collision detection/avoidance, and smart cruise control with vehicle-controlled lane changing.