Panopsis

Inspiration

Artur founded UkraineNow.org on the first night of the Russian invasion into Ukraine. We received 20k refugee evac requests in the fist days of the war, started connecting them to thousands of drivers willing to go into war zone and evac people. It was a challenging and risky environment because Russians were everywhere and there was a complete chaos in terms of a flood of information from all sources. Civilian videos, Telegram chats, Military comms, Enemy comms, etc. Same happened in actual military environment, Aleks (our advisor and friend) was in territorial defense defending Kyiv in the most hot places, then moved to eastern frontline within Armed Forces of Ukraine, they experienced the same challenges and actually keep experiencing it to this day.

What it does

Aggregation of multi-modal data sources (civilian data, military coms, enemy coms, OSINT)
Processing of source data into knowledge graphs with relationships
Grounding all the data in SPOTREP format
Allowing command and units to "chat" with your SPOTREP and underlying data

How we built it

We used multi-modal LLMs to process sources, built NER and triplet extraction pipelines, created knowledge graphs, visualized them and used LLMs grounded within SPOTREP and source data to allow custom inference.

We also used PRIMER & Vannevar datasets to enhance knowledge graph with relevant news stories and did RAG pipeline for that with Mongo+Qdrant vector store.

We used SkyFI for satelite imagery and used Radiant for LLM inference. We used Vercel to deploy the app and provide UI for chatting with your SPOTREP.

Challenges we ran into

Deployment, data processing, prompt engineering, complex UX.

Accomplishments that we're proud of

It actually works :)

What we learned

We learned what SPOTREP was and why it exists haha. And that there is a huge potential for this to exist based on our interactions with military people.