MEMENTO - Platform for Real time urban experiences

---

How did it began - The Inception of MEMENTO
Flashbacks that uncover its evolution and the journey from concept to creation.

What if we could gamify the city experience?

This was the germ seed of the idea — a simple, playful thought at its inception. The intention was to explore how I could combine the tools, methods, and skills I was being exposed to in the Computational Design Practices program at GSAPP in Columbia University— computation, design, AI, data-driven mapping — to create something that could reshape the way people experience the city in their everyday lives. How do we feel the city, touch it, smell it, see it? Could our daily commutes become interactive, sensory-rich experiences that deepen our connection with the urban environment?

To explore this further, I developed a documentary research project during the fall semester, where I began investigating my own relationship with the city — through self-documentation exercises, behavioral observations, and analytical mapping. The aim was to understand the problem statement through lived experience: breaking it down into key factors like body pattern, gaze, time, activity, and location, and examining how they interrelate. These insights laid the groundwork for drawing meaningful conclusions — ultimately leading to the creation of the platform: MEMENTO.

Have you ever started a journey,
reached your destination,
but couldn't remember how you got there?

The city felt like a blur,
lost in OBLIVION,
and the experience was disconnected —
as if the journey never really happened.

Through this process, I began to observe a troubling pattern: In today's technologically driven world, our relationship with urban environments has drastically changed. Modern commutes have become monotonous and disconnected, defined by passive interactions with the city. Once moments of observation and exploration, commutes are now dominated by screens, reshaping attentional habits and muscle memory. Gaze patterns, once directed outward toward the environment, now predominantly focus downward, reducing the city to a mere backdrop. This detachment diminishes meaningful engagement with urban spaces, eroding their historical, cultural, and emotional significance.

This realization led to the identification of a core problem !

---

Oblivion
the state of being unaware or unconscious of what is happening

An observational study on the disconnect between people and the city, exploring gaze patterns, physical interactions, and the relationship between time, location, and activity during urban commutes.

The exploration of body patterns, gaze, and time highlights how modern commuting behaviors have transformed our relationship with urban environments. Technology has reshaped physical habits, with body patterns reflecting detachment and routine, while gaze has shifted from outward exploration to a default downward focus. Similarly, time, which once fostered instinctive engagement with surroundings, is now compressed and hurried, leaving little space for meaningful connections. These shifts reveal the growing disconnection between people and the city.

Location and activity, however, offer pathways to reclaim this lost engagement. Location holds the power to shape interactions, encouraging discovery and connection, while activity remains a deeply personal factor that transforms commutes into unique journeys. Whether planned, spontaneous, or city-driven, activities allow individuals to build relationships with their surroundings, turning mundane routines into opportunities for exploration. Together, these factors present the potential to reconnect individuals with urban environments, transforming disconnected commutes into meaningful urban experiences.

---

Climb the steps of Odyssey, leaving behind the monotony and disconnection of Oblivion to rediscover the city through meaningful engagement and exploration.

---

Odyssey
a long and eventful or adventurours journey or experience

A speculative journey designed for those who want more than a commute, it is for people seeking to connect, engage, interact, and rediscover the city, transforming commutes into engaging, discovery-filled journeys and turning the city into an interactive playground.

Odyssey offers a counterpoint to Oblivion — a reframing of the commute as a meaningful journey. It emphasizes awareness, engagement, and discovery, challenging the assumption that everyday urban routines must be monotonous or unconscious. Instead, Odyssey proposes that even the most ordinary routes can become spaces for reflection, exploration, and interaction with the city.

The idea of Odyssey emerged from a simple yet pressing realization: that digital saturation is dulling our connection to the physical world. As we increasingly experience cities as blurred scenery behind a screen, the Odyssey state calls us to re-tune ourselves to the emotional, sensory, and cultural layers of our surroundings. This isn't a rejection of technology — it's a reorientation. Odyssey leverages technology not to distract, but to enhance presence.

At stake is the growing disconnection between people and place. Odyssey reimagines commutes not as empty transitions, but as opportunities for participatory discovery. It encourages users to become active agents in their environment — transforming passive routines into moments of curiosity, attention, and meaningful interaction with urban life.

---

Urban Experiences as Mementos
Deconstructing the Urban Experience — to Understand How It Forms, Functions, and Feels.

Urban experiences emerge from the dynamic relationship between people, the city, and the experiences that unfold in between. These entities do not exist in isolation — they are constantly activated through three core actions: intersection, interaction, and interplay.

• The Actions: How We Engage

• Intersection - occurs when people and places meet — in motion, in pause, in fleeting moments of presence.
• Interaction - happens when the city responds to our actions and we respond back — visually, sensorially, socially.
• Interplay - is the continuous, evolving dance between the environment and the individual — a layered exchange of meaning, attention, and response.

Together, these actions shape the nature and quality of the urban experience. They determine whether we pass through the city blindly — or engage with it fully.

• The Entities: Who and What Create the Experience

• People - bring mood, memory, routine, and perception.
• The City - offers setting, stimuli, rhythm, and unpredictability.
• Experiences - form in the space between — ephemeral, emotional, and deeply personal.

When these entities connect through mindful action, a new kind of journey unfolds. The city becomes more than a backdrop; it becomes a collaborator.

This visual represents one of the early commute archives conducted as an experimental framework to collect real-world user data. Through a structured set of tasks — including path discovery, photo capture, object study, smell and sound mapping, journaling, and mood reflections — diverse inputs such as geolocations, images, sensory notes, and personal reflections were gathered.

This experiment was not just about documentation — it was about decoding how urban experiences unfold during commutes and how they can be meaningfully captured on a digital platform. The archive played a critical role in shaping MEMENTO's design: informing its structure, defining input types, guiding feature development, and mapping how user-generated data could translate into live, discoverable mementos. It also laid the groundwork for categories, visualizations, and interaction models that the platform now enables.

The research revealed that urban detachment doesn't stem from a lack of meaning — but from a lack of active engagement. By paying attention to where we are, what we do, and how we perceive, we begin to reclaim a sense of urban presence. Urban experiences are not just fleeting moments — they are powerful tools for transforming Oblivion into Odyssey, reminding us that every step through the city holds the potential for discovery.

---

What is MEMENTO?
Understanding the characteristics of MEMENTO the platform.

MEMENTO reimagines how we engage with the city — not as passive commuters guided by algorithms, but as attentive explorers tuned into real-time urban experiences. The platform shifts our focus from screen-based distraction to place-based presence. It's not about checking in, scrolling through feeds, or arriving at a destination. It's about noticing what's unfolding around us — the sounds, sights, encounters, and emotions that make each journey unique.

While most digital platforms demand attention for virtual content, MEMENTO gives that attention back to the physical world. It doesn't reward perfection or popularity — it values presence. What other apps ignore or overlook, MEMENTO elevates: a street-side moment, a passing gesture, a fleeting vibe. This platform isn't built for likes or followers, but for those who want to feel the city, not just move through it.

What MEMENTO Is (and Isn't)

• Not another navigation app — but a platform to re-navigate how we notice the city.
• Not just a place to document what's seen — but to create your own urban moments as mementos.
• Not an app to show destinations — but one that celebrates the journey itself.
• Not designed for screen time — but made for city time.
• Not about permanence or perfection — but about capturing what's fleeting, raw, and real.
• Not another tool to scroll through — but one that invites you to walk through.
• Not a social app for likes — but a quiet space to reflect, feel, and connect.
• Not another place to record for later — but to notice what's happening right now.
• Not a map of what's already known — but a living canvas of urban experiences unfolding in real time.
• Not what other platforms consider 'valuable' — but everything they overlook.
• Not another city app for efficiency — but a gentle reminder to look, slow down, and feel.
• Not built to distract you — but to anchor you to the here and now.

---

WHAT ARE THE Mementos?
Ephemeral urban experiences — captured, shared, and rediscovered.

Urban experiences include ephemeral encounters, emotional reflections, personal connections to places, and the unseen layers of the city. They span from spontaneous events and mood-driven spaces to unmapped details, everyday activities, and ambient moments that shape our relationship with the urban environment. These experiences, often overlooked by conventional navigation platforms, are what make the city feel alive and personal. MEMENTO brings these elements to the forefront — capturing and sharing them as meaningful traces in a collective urban memory.

• Urban Experiences as Mementos
  Capturing the fleeting, meaningful, and often overlooked moments of the physical world.

Urban experiences span a wide spectrum — from spontaneous street performances and pop-up events to unplanned social gatherings, niche discoveries, atmospheric changes, and moments of solitude or surprise. They include not only visual phenomena but also smells, sounds, moods, memories, and personal reflections tied to specific places. A fleeting scent, a protest on a street corner, an unexpected mural — all become mementos that define the emotional rhythm of the city.

These are experiences that surround us every day, yet remain invisible to most digital platforms. They aren't part of formal city datasets or commercial navigation apps. MEMENTO fills this gap by surfacing what is unmarked, ephemeral, or emotional — capturing the human side of the city that's often lost in the rush. It's not about guiding you to a destination, but letting you discover what makes the journey meaningful.

• memento categories - What They Mean
  MEMENTO captures the city's intangible layers through 20 unique categories, each representing a distinct type of urban experience — from fleeting encounters to cultural markers, and everyday interactions to hidden stories. Here's what each category means:

• memento tags-What They Represent
  MEMENTO tags add a layer of emotional, sensory, and contextual depth to each captured moment, transforming fleeting experiences into meaningful stories. Here's how each tag frames the city's intangible elements:

• memento duration-Time as a Layer of Experience
  MEMENTO durations capture the temporal essence of each moment, framing experiences through time — from fleeting encounters to long-lasting impressions. Here's how each duration shapes the memory of a moment:

• Types of Mementos - Datasets based on the creators
  Understanding how mementos are created and shared on the platform

MEMENTO supports three primary types of urban mementos that reflect different forms of contribution and discovery:

Together, these types allow MEMENTO to become a truly collaborative, living archive — built for, by, and of the people and the city.

• Mementos on the Platform MEMENTO
  Snapshots of moments shared and experienced through MEMENTO

Displayed here are a selection of mementos created and shared on the MEMENTO platform. Each one represents a real urban experience — discovered during daily commutes, marked by emotion, place, and time. From fleeting street performances and curious signs to overlooked public details and quiet moments of reflection, these mementos reveal the hidden richness of the everyday city.They serve as a glimpse into how users interact with MEMENTO: observing, capturing, and contributing to a collective archive of lived urban moments.

---

How MEMENTO Works?
Understanding the Platform and What It Offers

• 3 Core Actions of MEMENTO

MEMENTO functions through three core actions that guide how users interact with the platform. Together, these actions form a continuous cycle of participation, enabling users to create, discover, and interact with real-time urban experiences.

1. Capture Users can add their urban experiences as mementos — fleeting moments they encounter on their commutes. Each memento includes media (images or videos), a short reflection text, location and time details, and a selected nature or category. These inputs ensure the memento is discoverable by others.
   What is created becomes something for others to discover.

2. Discover Users explore the mementos available on the platform using filters, personalized settings, curated recommendations, and spatial browsing. This enables each user to access real-time, nearby urban experiences shared by others.
   What is discovered leads to meaningful engagement.

3. Interact The platform encourages users to engage with the mementos not just digitally, but physically — by visiting, experiencing, or responding to the moments they discover. This interaction creates a feedback loop that leads users back to the act of creation.
   What is engaged with can inspire the next memento to be created.

---

• Role of Computation and Machine Learning in MEMENTO

MEMENTO is not just a platform — it's a computational ecosystem that leverages a wide spectrum of tools, methods, and data-driven processes to transform everyday urban experiences into interactive, real-time narratives. It integrates multiple computational layers, each playing a distinct role in capturing, analyzing, and visualizing the city's overlooked moments.

MEMENTO's computational framework is divided into two core sections:

Together, these front-end and back-end components create a cohesive computational ecosystem that transforms urban experiences into interactive, real-time narratives, making the city's overlooked moments discoverable and engaging.

---

• Computational Workflow

MEMENTO's workflow is structured around the creation, visualization, exploration, and curation of urban experiences, transforming scattered moments into structured datasets. The process includes both user-generated and public mementos, creating a dynamic, interactive map of the city's fleeting encounters.

1. Creation of User Mementos
   Users upload media, add text reflections, and tag locations, creating structured mementos categorized by type, tag, and duration.

• Media, text, and geolocation inputs.
• Categorized using predefined lists (categories, tags, duration).
• Data stored in Firebase as structured memento entries.

2. Creation of Public Mementos
   Public mementos are generated through data scraping and machine learning, integrating citywide events and activities as real-time mementos.

• Data sourced from public platforms.
• Machine learning processes data in real time.
• Structured to align with MEMENTO format.

The Machine Learning pipeline for public mementos in MEMENTO is designed to automate the classification and tagging of public content sourced from platforms such as Secret NYC, Reddit, NYC Bucket List, Newsbreak, and more. The objective is to replace rule-based keyword matching with a robust multi-label classification model that leverages existing user-generated mementos as training data.

The pipeline consists of three primary models — Category Classification, Tag Prediction, and Duration Estimation — each trained using supervised learning techniques. Logistic Regression with One-vs-Rest strategy and Random Forest classifiers are employed to predict relevant categories and multiple tags, while Decision Trees handle ordinal duration estimation. Text data undergoes preprocessing, including tokenization, stop word removal, and TF-IDF vectorization to convert descriptions into feature vectors suitable for model training.

By implementing this ML pipeline, MEMENTO enhances the consistency and accuracy of content classification, enabling the platform to transform unstructured public content into structured, contextually enriched mementos that align with its existing taxonomy.

3. Mementos on MEMENTO
   All mementos — user and public — are populated on a real-time interactive map, each tagged with location, media, timestamp, and description.

• Data visualized on a dynamic map.
• Displays geolocation, media, and descriptive tags.
• Serves as a playground of real-time urban experiences.

4. Exploration of Mementos Users explore mementos using filters and settings, discovering experiences by category, tag, duration, and proximity.

• Filter by categories, tags, duration, and distance.
• Explore mementos through curated lists and live feed.
• Discover mementos based on user's current location.

5. Curation of Mementos Recommendations are generated based on user profiles, interaction history, and data analysis, creating personalized memento lists.

• Personalized memento curation.
• Daily, trending, recommended, and nearby mementos.
• Data-driven recommendations based on user behavior.

More about the Computation tools, methods and Machine Learning Pipeline in MEMENTO

---

• User Types Understanding MEMENTO Through Types of Users and Their Journeys

Not all city explorers are the same. MEMENTO recognizes this diversity and adapts to different user types — from casual observers to detail seekers.This section presents four user archetypes, each with a unique way of engaging with the urban environment. For every user, we explore:

• Their behavior before using MEMENTO — often marked by distraction or routine (Oblivion)
• The shift or trigger that prompts them to try MEMENTO
• Their new behaviors and experiences after using MEMENTO (Odyssey)

These journeys show how MEMENTO becomes a personalized companion, turning passive city routines into active, emotional, and participatory experiences.

a. Explorer 1: The Capturer

Before MEMENTO:
This user drifted through the city on autopilot — camera in hand, yet disengaged from their surroundings. Photos were taken for the sake of memory, not meaning. The city existed more as a backdrop than a lived environment. Their commute path remained fixed — focused solely on leaving the origin and reaching the destination, with no attention paid to the urban experiences along the way.

With MEMENTO:
The Capturer began to actively record fleeting urban experiences — tagging moments with emotional tone, time, place, and category. Using features like photo/video upload, reflection text, memento tags, and categorization, they transformed casual snapshots into deliberate urban stories.

After MEMENTO:
Their commute evolved into a daily ritual of observation and reflection. The Capturer's gaze lifted from the screen to the street. What was once passive scrolling became mindful engagement — turning each walk into a visual and emotional exploration of the city. Now, they organically change paths based on nearby mementos and real-time urban experiences. Even if it means a longer route, the commute is no longer about reaching the destination, but about discovering moments along the way.

b. Explorer Type 2: The Visual Explorer

Before MEMENTO:
This user spent hours scrolling through Instagram, bookmarking aesthetic spots, yet rarely visiting them in real life. Their connection to the city was mediated entirely through screens — visual, filtered, and detached. Commutes were passive; paths rarely changed, and the urban world was consumed more as digital content than lived experience.

With MEMENTO:
They began using the discovery feature to surface nearby mementos based on category, tag, or mood — using filters like radius and type to find things aligned with their interests. Instead of saving places for "someday," they now saw live mementos from real users as invitations to explore the physical city.

After MEMENTO:
The Visual Explorer now walks with intention — making spontaneous detours to follow mementos, visiting places they might have once ignored. The phone is no longer just a viewer but a bridge to engage with physical reality. Their route is no longer fixed but guided by curiosity and presence. Digital discovery now fuels real-world interaction, not just virtual consumption.

c. Explorer Type 3: The Niche Seeker

Before MEMENTO:
The Niche Seeker relied heavily on generic travel blogs, mainstream guides, and top 10 lists—yet often felt uninspired. Their commutes were passive, with limited excitement or discovery. They struggled to find places that resonated with their unique interests, style, or emotional tone. Even in a city full of culture, their experience felt filtered and flat.

With MEMENTO:
Using MEMENTO's tag and category filters—like "Hidden Gem," "Niche/Cult," or "Emotionally Charged"—this user began discovering urban moments tailored to their sensibilities. The platform's personalized feed offered micro-experiences that weren't visible on mainstream platforms, allowing them to explore the city through a more authentic and individualized lens.

After MEMENTO:
The Niche Seeker now curates their own trail of mementos — a hyper-personalized map of meaningful spots. Their relationship with the city has become more expressive, reflective, and intuitive. Instead of consuming what's trending, they now discover what truly aligns. Even routine paths are now layered with purpose, emotion, and surprise.

d. Explorer Type 4: The Busy One

Before MEMENTO:
Always in a rush, The Busy One moved quickly between meetings, errands, and obligations. Phone in hand, but never used to engage with the environment. No time to pause, reflect, or capture. Nothing saved, nothing explored. The city was just a blur between point A and point B.

With MEMENTO:
MEMENTO's lightweight features—like curated suggestions, real-time live feed, and quick capture options—fit seamlessly into their tight schedule. Without needing deep interaction, this user could still mark moments, save spontaneous experiences, and stay aware of what's happening nearby, even in seconds.

After MEMENTO:
Now, the Busy One adds moments effortlessly—turning mundane stops into moments of discovery. Even short breaks become opportunities to engage. Their fast pace remains, but now includes awareness, memory, and joy. Commutes become more intentional, dotted with quick but meaningful encounters with the city.

---

---

End? Or the real Beginning!
This isn't the conclusion of a project —
it's the start of a movement.
A shift in how we move through our cities,
how we notice, reflect, and reconnect.

Cities don't lack stories.
We've simply stopped paying attention.

MEMENTO is a gentle reminder:
to look closer, feel deeper, and share what usually goes unnoticed.
It's not a tool. It's a mindset.
Not just a platform, but a call to presence.

Thank you for being part of this journey —
from the quiet blur of Oblivion…
to the rich discovery of an Odyssey yet unfolding.

---

Who's Behind MEMENTO ?

Vaibhav Jain - created and designed by
Computation | Spatial Data | Architecture
Designer | Storyteller

MEMENTO is a culmination of my academic and creative journey through the Computational Design Practices program at GSAPP in Columbia University. Over three semesters of research, experimentation, and refinement, MEMENTO was shaped by critical feedback, collaborative learning, and speculative design thinking.

Core Studios and professors That Anchored the Project
Explore, Explain, Propose – Laura Kurgan, Snoweria Zhang
Design in Action – Seth Thompson, Catherine Griffiths
Capstone Project Advisor – Juan Francisco Saldarriaga

Special Thanks To
My peers, mentors, and the entire CDP cohort for the countless conversations, provocations, and encouragement that fueled this work. Every discussion, critique, and shared idea contributed to the making of this platform.

---

References

• Secret NYC
• Walk the City – Book by Rosemary Dawson
• Pokémon Go
• Gouda Gamified Tour – Peek
• StoryTourist
• Kresna: A Game and Cultural Tourism System
• Senseable City Lab – MIT
• Gamification in Museums – AMT Lab
• Domestic Robocop – Keiichi Matsuda
• Augmented (Hyper)Reality – Vimeo