The former Annie Lytle Elementary School by I-95 between Riverside and Brooklyn has long had a reputation as the “most haunted place in Jacksonville.” With urban myths abounding, the abandoned landmark has become a mecca for young adults seeking supernatural adventure. Here’s the story of the mysterious old school, how it became a fixture of First Coast folklore, and the people trying to save it for the generations to come.

Schoolhouse myths

The Jacksonville area teems with ghost stories and urban legends, from spaces said to be haunted like Kingsley Plantation, the TacoLu building and Evergreen Cemetery to tales of strange happenings like the Orange Park Humanzee and mysterious Betz sphere of Fort George Island. But nowhere stands out in the annals of First Coast folklore like the foreboding Classical Revival schoolhouse located beside the I-95 interchange in the Brooklyn neighborhood. Known as Annie Lytle Elementary School and, as the inscription above the entrance has it, Public School Number Four, the building has lain vacant since the 1970s. 

The school’s long afterlife has attracted all manner of legends. Visitors have claimed the voices of children long since departed still echo in the ruins, and a number of wildly varied stories circulate about it, each more outlandish than the last. The most common variant tells of schoolchildren killed by a boiler explosion, while other stories attribute the children’s death to a psychotic janitor who went on a killing spree. Even more absurdly, a third version blames a cannibal principal who devoured delinquents sent to the office. In the 1980s and ’90s, the age of heavy metal and moral panics about satanic cults, the building gained a reputation as a haven of diabolical activity, earning it the nickname “The Devil’s School.”

There’s no truth to these stories. They mostly function as an impetus for a common teenage activity: “legend-tripping,” the right of passage of braving locations associated with frightening legends for purposes of initiation, rebellion or simple entertainment. Annie Lytle Elementary is Jacksonville’s premier location for legend-tripping, surpassing similar locations like the erstwhile Ghost Light of Greenbriar Road and Fernandina’s Wiccademous grave.  

This association has led the school to be featured in countless articles, websites and books about haunted places over the years. But how did an abandoned elementary school become such an endemic part of the folk life of the city? Where do these crazy stories come from? To answer that, we need to look at the history of the building and what has led to its long abandonment.

The story of School Four

The school’s origins date to 1891, when Riverside had emerged as a fast-growing neighborhood of Jacksonville. The city built a small wooden schoolhouse for white children, originally named Riverside Grammar School or simply School Four, at the southeast corner of Gilmore and Charles streets (now Peninsular Place and Chelsea Street). At the time, it was across the street from Riverside Park, which then extended all the way to Gilmore Street.

The current building dates to 1917, a time when the city considered public education such a lofty imperative as to demand monumental architecture. In 1915, Duval County voters passed a $1 million bond issue to build 12 new sturdy brick schools; Riverside Grammar School, having already outgrown its building, was one of the first constructed. By 1927, Riverside Grammar was the county’s second-biggest school with 775 students.

The new building’s architect was Rutledge Holmes, who relocated from Charleston to help rebuild the city in the aftermath of the Great Fire of 1901, designing iconic local buildings including the Holmes Block and the Seminole Club, now Sweet Pete’s Candy. Holmes’s last years were tragic. His wife died in 1925 and he later underwent mental health treatment at Florida State Hospital. Subsequently he moved to the Panhandle and set up an office in Quincy. On May 2, 1929, he took his own life in his office after drinking heavily, leaving behind a note reading:

“Do not notify anyone. I have some pains in the region of my heart. Should I die I would like to be wrapped in one of my camping blankets and buried under some pretty trees in the country in an unmarked grave. Take what I have in Quincy for the trouble.”

The school quickly emerged as a neighborhood landmark. “I have loved that school since I was a child,” says Patsy Bryant, an expert on the school and a member of the Annie Lytle Preservation Group. “We would ride by there on the way to my aunt and uncle’s house, and it just fascinated me. Many of us are like that.”

Educator Annie Lytle, later Annie Lytle Housh, served the school as teacher and principal for most of its early existence. A native of Yellow Springs, Ohio, Lytle moved to Jacksonville with her family in 1883 and started teaching at Riverside Grammar when she was just 18. She was named principal in 1914, before the construction of the new building. “She lived right there on Park Street,” says Bryant. “She could watch that building being built, and I’m sure she was over there giving her input, making sure they did it right.”

“Miss Annie” was remembered as a stern disciplinarian who had the students line up and march into school each day with John Phillip Sousa marches playing on a Victrola. Bryant says her strictness was legendary among the children. One former student, Susan, told Bryant Lytle was “real short and they called her ‘Old Pruny Face.’” Lytle retired in 1949 after 35 years as principal, and the next year, Annie Lytle Elementary School was renamed in her honor. She died in 1957 and is buried at Evergreen Cemetery.

The school’s end was hastened by the construction of what became I-95. Starting in 1954, Jacksonville began work on the Fuller Warren Bridge, part of the burgeoning expressway network that marked a new epoch of car-centered development. In a notable example of Jacksonville’s Godzilla strategy of urban development, expressway routes were partly chosen to facilitate the demolition of supposed blighted areas and to serve as physical barriers between Black and white neighborhoods; in Annie Lytle Elementary’s case, the bridge separated white Riverside from largely Black Brooklyn. The expressway also took out part of Riverside Park and cut the school off from the neighborhood it served, and just months after the Fuller Warren opened in March 1960, Annie Lytle Elementary’s classroom bells rang for the final time.

For the next decade, the building served as offices and storage for the school district until it was shuttered in 1971. The district sold the property to Central Christian School in 1975 but bought it back just five months later when the private religious school failed to obey federal desegregation requirements. After several unsuccessful sales bids, the Ida Stevens Foundation purchased the school to convert it into a senior living center in 1980. This project too failed, and Annie Lytle Elementary entered a long purgatory of dereliction and redevelopment plans deferred. Despite its good bones and fame, the construction costs, further widening of I-95 and plain bad fortune thwarted attempts to turn it into apartments and condos in the 2000s and 2010s, and the building has sat vacant for more than four decades.

Legend tripping with Annie Lytle

If the incursion of I-95 brought on Annie Lytle Elementary’s closure, it’s also in one key way responsible for its long afterlife: proximity to the highway ensures the school’s ruins are seen by thousands every day. For young Jaxsons, the formidable edifice is perennially foreboding yet highly visible, firing up many a youthful imagination and ensuring that well-placed graffiti will draw countless eyes. As such, the old schoolhouse emerged as Jacksonville’s legend-tripping spot par excellence.

Folklorists have increasingly studied legend-tripping since the 1970s, and note that the practice tends to follow a specific three-party pattern. In his book If You Should Go At Midnight: Legends and Legend Tripping in America, sociologist Jeffrey S. Debies-Carl calls these the “preliminal,” “liminal” and “postliminal” stages. In the preliminal stage, trippers begin their journey by recounting legends about the destination. Like Annie Lytle Elementary’s wild tales of deadly fires and murderous school staff, these legends require no basis in reality or consistency from one telling to the next. Their true function is to lay out a compelling reason for trippers to investigate the space for themselves and to set the appropriate mood for a voyage into the tenebrous unknown. 

Next comes the liminal stage, where trippers enter their destination and cross into the realm of unedifying uncertainty. Trippers enact rituals or activities designated to bring contact with the mysteries of the space, giving them evidence as to whether the legends are true, or at least melting their resolve enough that they flee the space in a frenzy of adrenaline. As nearly any photo of Annie Lytle School over the last decades show, enactment there often involves graffiti tagging. Finally, departing trippers enter the post-liminal phase as they return to the quotidian realm. The uncanniness of the experience lingering even in the safety of home, they craft their personal narrative of what they just faced, engaging in a “telling of the tale” that weaves their names into the fabric of the legend and keeps it alive for trippers to come.

Many Jaxsons have such tales for the telling about Annie Lytle Elementary. James Shannon snuck in around 2003 in his early ’20s. “It’s one scary place and the only time (to this day) I’ve ever had a paranormal experience,” he says. “We were on the stage in the auditorium and could hear dozens of kids’ voices… unmistakable and clear as night. Now, mind you, we had already explored elsewhere and were the only humans in the building.”

“I love Annie!” says local author Melanie S. Pronia. “My husband and I are both local horror writers and got married in front of it on October 31, 2014. The school is actually what brought us together… it’s an amazing building!”

Patsy Bryant, meanwhile, has never had one supernatural encounter in all her years of work. “People would rather believe the scary stories than the serious things,” she says. “I would go up there almost every day, lock myself in and go to work. And there ain’t no ghosts in there. I’m not saying I don’t believe in spirits – but not there.”

One story from writer Tim Gilmore of Jax Psycho Geo elucidates how these legends spread and evolve – and how Annie Lytle Elementary came to be dubbed the Devil’s School. He went twice with friends as a teenager around 1991.

“Some things that happen when you’re that particular age that just kind of become your own mythology, your own mythos, you know? Going into School Number Four was that for me.” Gilmore doesn’t recall hearing any of the now-common ghost stories about the building before his first legend-trip, and certainly nothing about devil worshippers. On one trip, an artistically talented friend had the idea of spray-painting different rooms with themes. “He did like a Volkswagen Bug room – he drove a Bug – and we did a Peace Room, with just like flowers and stuff like that. But one of the rooms we did with flames painted on the walls and we called it the Hell Room.”

Gilmore was perplexed some months later when a local news station did a story on Annie Lytle featuring footage of the “Hell Room.” “I remember the camera panning slowly across the flames. The news report was that our graffiti was evidence of satanic activity going on at the school.” This was news to Gilmore, who grew up a fundamentalist Christian, but this was a time of the Satanic Panic, a media-abetted conspiracy theory that purported that a secret cabal of devil worshippers was committing nefarious crimes across the country as part of their dark rites.

“We were very annoyed by it, like they stole our vision or something, but also weirdly proud,” says Gilmore. “Something we did make the news! Of course they didn’t show the Volkswagen Bug room or the Peace Room.”

Saving Annie Lytle

Gilmore and co weren’t alone in leaving graffiti that pearl-clutchers interpreted as clear-cut evidence that the school was an active temple to the dark lord Lucifer. By the time I made my first trek to Annie Lytle in 1999, this had become an indelible part of the mythology, and legend-tripping was drawing in so many bored young people each weekend night it might as well have been Adventure Landing or Einstein A Go-Go. Shortly thereafter, the school had an explosion of coverage by websites, local articles and books on mysterious locales that continues to this day. Unfortunately, for as much as Annie Lytle meant to local legend-trippers, the years of vandalism, weather damage due to busted windows and two fires have taken a heavy toll. In 2005, after the latest effort to redevelop the structure failed, the owners explored demolishing it altogether.

Then a group of do-gooders entered the scene – and offered a way to explore the structure while helping preserve it. Local advocate Tim Kinnear was opposed to demolishing the school, and took it upon himself to mow the grass and tend to the property. “Don’t let its derelict look and state fool you into believing it is going to collapse,” says Kinnear. “It was designed after the Great Fire to be fireproof.” Working with the owners under a hold-harmless agreement, he established the Annie Lytle Preservation Group, a band of like-minded preservationists who organized cleanups to remove debris, clean off the graffiti and stabilize the building. Patsy Bryant was one of the earliest members. “I started talking to Tim and I loved him from the minute,” she says. “We would go up there and work, and we started getting other people to come up there too.”

Sadly, the fate of Annie Lytle School is as much in limbo now as when the School Board exited its storied grounds in the 1970s. After years of failed projects, a complicated series of property transfers has left the building under a separate ownership than the surrounding land, which still belongs to the Ida Stevens Foundation. The preservation group has become far less active as Kinnear, Bryant and others have had to step away from the regular cleanups. Today, complete or even partial renovation seems remote. 

“It is not impossible if the right minds with the adequate money resources could persuade the present owners to somehow merge this property back together,” says Kinnear. “There are countless possibilities for the school… it will always be my hope that somehow, some way this place will be spared and repurposed.” “I just want it fixed back where people can go in and just see it the way it should be,” says Bryant. “Even with the mess of it, it is a beautiful building.”

For now, Annie Lytle Elementary School still stands, as it did when Miss Annie walked its halls, awaiting its chance at the renewal now sweeping Brooklyn.