Dropbox Sign Voldemort

An AI assistant, powered by Alexa and ChatGPT, simplifies document comprehension by reading, summarizing, and translating your Dropbox agreements.

Comment

Inspiration

Fun fact: every time we met to work on the project, we would inadvertently say 'Alexa,' triggering it. This evolved into us referring to it as 'he who shall not be named,' and eventually shortened to 'Voldemort.'

Problem

How many times have you signed an agreement without fully understanding its contents with confidence? Agreements are frequently laden with jargon, overly formal language, and pages upon pages of lengthy text. During our research, we were shocked to discover just how challenging the process of comprehending agreements can be, especially for individuals with blindness, those with motor skill difficulties, or those for whom English is a second language. Our goal was to transform the document signing experience with Dropbox Sign API into an accessible one for everyone. We will outline the problems we found from research in more detail below.

Lack of Accessibility for Some Disabilities

According to the World Health Organization (WHO), approximately 15% of the world's population, which amounts to roughly 1 billion people, experience some form of disability. To effectively address the challenges associated with document signing, it is crucial to prioritize accessibility and ensure equitable access for disabled populations.

People who are blind, have low vision, or experience certain motor disabilities encounter difficulties when it comes to comprehending agreements due to visual barriers, issues related to digital accessibility, or challenges when using traditional computer interfaces like keyboards and mice. Existing solutions (screen readers and special accommodations) available to people with disabilities often involve cumbersome bureaucratic processes, which can be time-consuming and limit a person's autonomy and flexibility when reviewing a document at their own pace.

Screen Reader Challenges

A screen reader is assistive technology software designed to assist individuals who are blind or visually impaired in accessing and interacting with digital content on computers. However, screen readers frequently present frustrating user experience issues, as they read aloud everything on a page, including non-essential elements, making it challenging for users to skim through content.

Special Accommodation Challenges

The process of finding and obtaining special accommodations imposes a burden on people with blindness and motor skill disabilitites. They lose both time and effort, along with their personal agency, which is required to find and secure reliable accommodations from government services or organizations for document reviews on their behalf.

What it does

While our initial purpose in developing Dropbox Sign Voldemort was to cater to the requirements of individuals with visual and mobile disabilities, as well as those for whom English is a second language, it has evolved into a universal solution that will enhance the contract experience for nearly all users.

Dropbox Sign Voldemort provides 4 features:

  • Overview of of contract queue and option to select which document to open
  • Read contracts verbatim
  • Summarize contracts in a digestible way
  • Translate documents into other languages or dialects like a “Gen-Z” dialect

Real Use Cases

A user can request Alexa to open and read a contract verbatim. If they are confused by the contract’s language, they can simply ask Alexa to provide a summary of the contract.This is particularly useful when a contract is laden with jargon and formal language, making it challenging for the user to comprehend.

Additionally, when a contract is not written in the user's first language, they can ask for it to be translated into their preferred language. Dropbox Sign Voldemort goes a step further by offering translation into specific dialects. For instance, if a Gen-Z user struggles to understand the contract's formal language, they can request an explanation in a Gen-Z dialect.

How we built it

We employed Alexa, a Natural Language Processing (NLP) system, to manage voice recognition and transcription services. NLP models powered by ChatGBT were responsible for tasks such as translating various languages and dialects and summarizing documents. To facilitate communication between Dropbox Sign and ChatGBT, we developed an API middleware using Flask. Within this middleware, we utilized Chat GPT endpoints to execute document summarization and translation tasks. We deployed our Flask application using AWS Elastic Beanstalk. We also took advantage of the S3 storage that comes liked to an Alexa Skill via the Alexa Developer Console as a type of "cache".

We established multiple endpoints through which our API interacted with Dropbox Sign to retrieve the following information:

  • Account Info
  • A list of documents that have been signed or need to be signed
  • Specific documents
  • The file download url
  • The text from a document. We used a library called PyPDF2 to read and extract text from the PDF.

Challenges we ran into

Deploying a Django application on AWS proved to be a challenging endeavor, leading us to pivot and transition our application to Flask for a smoother deployment process. Our efforts with the Alexa integration posed some difficulties as well, particularly in ensuring seamless document reading. Initially, our vision involved leveraging Computer Vision to identify signature placement on documents. However, we encountered hurdles during model training, signature placement, and integration with Dropbox Sign. While we had hoped to include a vocal authentication feature for signing, our timeline didn't allow us to develop it into our minimum viable product.

Accomplishments that we're proud of

We take pride in the substantial amount of time we dedicated to researching the problem space and understanding various user needs and pain points. Moreover, we are proud to be a team of engineers and designers who collaborated to brainstorm a solution collectively. Thanks to our team's diverse backgrounds, we leaned on each other for our unique strengths.

What we learned

We discovered the immense value of formulating a well-defined plan and establishing a minimum viable product (MVP) before embarking on the product development journey, especially considering our tight deadline and limited team resources. For some of us, it was our first time working with AI technology, making it a valuable learning experience.

What's next for “Dropbox Sign Voldemort”

In the future, we plan to invest more time into testing Dropbox Sign Voldemort with Dropbox users and to continue iterating and enhancing the user experience. Additionally, we aim to extend the compatibility of this technology to include other AI Natural Language Processing systems such as Google Assistant and Siri.

While we initially aspired to introduce a vocal authentication feature for signing (a nice to have), we were unable to incorporate it into our minimum viable product due to constraints related to our timeline and available resources. Nevertheless, our goal is to further develop and integrate this vocal authentication signing feature in future iterations.

Built with

We built Dropbox Sign Voldemort with a Flask backend API, AWS Elastic Beanstalk, AWS S3, an Alexa Skill, the Dropbox Sign and Chat GPT apis.

Built With

Share this project:

Updates