RemindMEET: Location Reminders based on Machine Learning

A mobile app that harnesses the power of AI technology to 1) easily create accurate location-based notifications, 2) remind you when to leave for an event, and 3) coordinate meetups and arrival times.

Comment

Inspiration

We’re forgetful people — approximately 56% of Americans consider themselves forgetful and 66% believe they have been more forgetful in the past decade. I can attest to this, as someone who regularly forgets to bring my wallet or send an email once I get home. This will continue to worsen are America experiences an aging population, making general forgetfulness an increasing part of our daily lives.

Additionally, forgetfulness are often associated with location-based tasks. For example, 51% of Americans reported forgetting things when they went grocery shopping. Many also report not remembering things when multitasking, especially in the digital world where we are constantly exposed to stimuli.

Statistics Source: https://nypost.com/2020/06/29/the-most-embarrassing-thing-to-forget-according-to-research/

What it does

While many see this as a serious inconvenience, I see it as an opportunity to continued integration of modern technological advancements to improve our daily lives. I have created is RemindMEET, a location-based reminder and social meetup app. It can be broken up into 3 sub-features:

1. Personal location-based reminders - RemindMEET allows users to set a reminder for themselves as soon as they enter a location. The location can be as large as a shopping mall or the size of a classroom. Users can customize a small message and reminder frequency (i.e. set time frame, specific days, etc). The next time the user enters the designated location, they receive a push notification. The novel geofencing algorithm, powered by AI, is able to function well with high accuracy even for small areas (tested in Wolman Hall, Library, and Hodson Hall). It is adaptable to different, irregular building sizes and lower power consumption compared to other apps using high-accuracy GPS. It is also able to improve and learn over time to improve its accuracy. RemindMEET also doesn't require the user to manually annotate the geofencing location, which is impractical and often results in poor accuracy due to GPS noise.

2. Personal event reminder - Users can also set reminders for attending events at different locations (i.e. go to airport, drive to mall, go to dentist appointment. RemindMEET sends reminds them when to leave based on up-to-date traffic information. It also supports different modes of transportation such as biking, walking, buses, subways, and railways. The app accomplishes this by harnessing Google Cloud's Distance Matrix API to ensure accurate, real-time notifications.

3. Meet-up Coordination & Arrival Time Tracking - This feature is a fusion of the previous two, enabling users to send RemindMEET meet-up requests to other RemindMEET users (i.e. their friends and family). RemindMEET provides each user in the group with everyone's elapsed time from the meet-up location and sends a notification when someone has arrive or is running late. Note: This is an envisioned feature from the start that has not been implemented yet. Though, the basis of this technology has already been implemented on individual user level through features 1) and 2).

All these features require the sampling of location services and has been tested to run in the background seamlessly and reliably.

How we built it

RemindMEET is built using Flutter providing users with a native app experience. Flutter is cross-platform and is easily scaleable to iOS devices.

RemindMEET's novel geofencing algorithm is one that is able to withstand GPS noise and irregular building sizes. It is machine learning model that is computationally lightweight, running directly on mobile devices with minimal latency and battery consumption. The model is a 2-part regression-based binary classification model that is probabilistic in nature:

1. Innate System - The innate system is a pre-trained model that performs the initial decision (or a bulk of the) decision-making process by assessing the change in distance to the target point designated by the user. In general, a user exhibits a particular movement pattern once they have arrived at the location -- its less "linear" and more "circular" or stationary in nature. After collecting 1500 data points of GPS samples (approximately at 6 second intervals) for movement away, towards, and at the target location, I concluded that a K-Nearest Neighbor (KNN) model was best suited due to its non-parametric nature that allows it to fit the non-linear data. The input x = { x1, x2, ... x5 }, where x1, x2, ... x5 are 5 consecutive distance measurements for the target point. The k value is set to an experimental value of 27. The model is able to perform well, correctly classifying "away" (out), "towards" (out), and "at" (in) with an accuracy of 80%.

2. Adaptive System - The adaptive system is a model that starts of "knowing" nothing and gradually learns over time. Its goal is to take in an input x = { x1, x2 } where x1 = latitude and x2 = longitude and predict in or out. Again, a KNN model is used due to its advantageous non-parametric nature for non-linear data and relatively low computational requirements with a dataset size <= 2000. The initial phase represents the learning phase, where the adaptive model simply learns the input/output of the innate model, in addition to other factors such as distance from point and in/out movement patterns. A balance of in/out datapoints is ensured. The next phase is the fine-tune phase, where the model is only trained when it misclassifies a data point. Gradually, as its dataset increases, it becomes more influential in the in/out decision-making process until it overtakes the innate model in becoming the dominant decision-making factor.

Challenges we ran into

There were 4 key challenges in developing the machine learning geofencing model for location-based notifications:

1) GPS is a course-grained measurement and is subject to a lot of noise, especially in indoor and urban environments

2) The size of buildings and locations are irregular in shape, not circular

3) High accuracy and frequent GPS polling results in high battery consumption

4) Uploading GPS data to the cloud is privacy intrusive

I overcome GPS noise by collecting noisier data for the KNN model to recognize. I also increased the sampling frame from 3 to 5 for the model to assess a larger range of data. Additional, I also introduced the concept of a "sliding window" that took into consideration whether the user was moving towards or away from the target point. The model can switch from "in" to "out" only if it consistently detects the user moving away from the target point. Conversely, the model can switch from "out" to "in" only if it consistently detects the user moving towards the target point. This allows the model to distinguish random GPS "jumps" across the block or if the user is simply passing by a location.

A naive approach of having a set distance from the target point results in poor performance due to the varying shapes of the geofencing area. Often times, the target point is also not centered, further reducing geofencing accuracy. The assessment of movement towards and away, as well as the implementation of RemindMEET's adaptive model helps combat the issue of irregularly sized buildings. Change in distance to the target point is irrespective of actual position and distance. Gradually, the adaptive model helps learn the shapes and boundaries of irregularly shaped buildings.

Constantly requesting high accuracy GPS data every 5 seconds was very battery hungry. To overcome this, I default to medium level GPS accuracy (block-wide accuracy) and intermittently switch over to high accuracy GPS for 2-3 samples at a time. This effectively lowered sampling frequency of high accuracy GPS data, significantly reducing power consumption.

Finally, instead of running our model on the cloud, I elected to implement it directly on the mobile device. With this in mind, I designed the model to favor simplicity over complexity, practicality over sophistication. Thus, individual location-based notifications are processed locally and are never sent to a server, making the approach privacy protecting.

Accomplishments that we're proud of

This was the first Hack-a-thon where I built anything remotely cohesive so I'm proud of the progress I've made. I'm glad that I'm gradually stepping up the level of sophistication for my CS projects and that I came in here with a well-fleshed plan instead of flying blind. I'm happy with how I processed, tackled and overcame the unexpected challenges I encountered. Additionally, the UI turned out quite good considering it has always been by achilles heel in terms of app development.

What we learned

I learned how to tackle challenges and problem solve under time pressure. I also learned how to formulate a more cohesive and solid plan of attack before beginning the coding process. Things that appeared simple and relatively straight forward often end up being more complicated, so I've learned how to adapt and make the necessary adjustments to my goals along the way (i.e. drop feature 3). I learned how to keep things simple, set realistic expectations, and better manage my time (when to code, when to stop, when to start working on report, etc).

What's next for RemindMEET: Location Reminders based on Machine Learning

I would like to gather more market validation on the RemindMEET. Furthermore, as described above in the features section, I would like to integrate features 1) and 2) to implement feature 3). I would also like to collect more concrete data about location based-notification accuracy and improve model detection performance, as well as how to further optimize power consumption. I would also like to implement the backend system with Firebase Authentication and Realtime Database for sharing of information between multiple RemindMEET users.

Share this project:

Updates