Goodman Community Center | Madison Black History: William & Anna Mae…

Madison Black History: William & Anna Mae Miller and Historic East Dayton Street

Remembering Madison's first Black neighborhood.

February 28, 2024 |

Do you know the history of the Millers and Madison's first Black neighborhood?

William Martin Miller, born 1872 in Kentucky, went to law school at Berea College in Kentucky. Unfortunately, it was also illegal for Miller to practice law in Kentucky as a Black man. Hoping for more opportunity in the north, Miller moved to Chicago.

Still unable to find a job in Chicago, Miller moved even further north to Milwaukee where he quickly got a job at the Plankinton Hotel as a waiter. One of his favorite customers, according to his grandson Billy McDonald, was “Fighting Bob,” aka Governor Robert M. La Follette.

After getting to know Miller and his journey from Kentucky to Wisconsin, La Follette knew that Miller's intellect, talent and dedication would be an asset to his administration, so he asked Miller to be the official Advisor to the Govenor.

William & Anna Mae Miller, pg. 17 of Settlin by Muriel Simms
Photo of page 17 from Settlin': Stories of Madison's Early African American Families.

Ecstatic about the new opportunity, Miller left for Madison on La Follette’s heels in 1901. Unfortunately, when he arrived his position of Advisor had been lowered to that of a ‘messenger’ by the state. Wisconsin would not allow Miller to practice law, even for the Govenor. But Miller remained committed to his high-powered friend, staying in Madison for the rest of his life and creating a long-lasting relationship between the Govenor, himself and his new neighborhood.

When he first moved to Madison, Miller lived in John Turner’s boarding house, which was also the Douglas Beneficial Society, at 649 E. Dayton, the site that would eventually become the longstanding Hill Grocery Store.

In 1902, Miller and Turner joined forces to build the African Methodist Episcopal Church (A.M.E), the first Black church in Madison and the home base for much of Madison’s Black activism in the early 20th century.

A few years after moving to Madison, Miller married Anna Mae Stewart, a Knoxville College educated teacher, in 1904. By 1908, the couple bought the two properties next to the Turner’s house: 645 and 647 East Dayton Street. The family lived in 647 and turned 645 into apartments exclusively for Black students and families moving up from the south. 645 E. Dayton was eventually demolished, but 647 remains standing and is marked as the oldest Black owned building in Madison.

The Miller House, 647 E Dayton St.
The Miller House. Wisconsin Historical Society

Beyond the buildings they acquired, both William and Anna Mae were deeply embedded in the community and the effort to create better lives and living conditions for Black folks in Wisconsin and around the nation. In 1909, Anna Mae Miller and her home worked in tandem to create the Book Lovers Club.

    “[The Book Lovers] club [was] designed to improve African American intellectual development by reading articles and poems by prominent Black people and discussing the issues of the day.”

    William was one of the original Niagara Eleven who started the NAACP. A group that included the renowned philosopher, scholar and activist W.E.B DuBois. Anna Mae was appointed a delegate to the National Congress of Colored Educators by Wisconsin Governor Francis E. McGovern in 1912.

    Douglass Beneficial Hall
    Douglass Beneficial Hall. Wisconsin Historical Society

    When William died of pneumonia in 1922, Anna Mae continued as the family matriarch, providing support for her loved ones and remaining active in the fight for racial equality. At 86, Anna Mae took part in the sit-in at the WI state capital to support the elimination of housing discrimination. Anna Mae died in 1963 at 88 years old.

    The development of the Miller’s two properties, the Turner House and AMEC made E. Dayton St. a budding and self-sustaining area. The buildings offered Madison's growing Black community space to worship, shop, gather, sleep, eat and create the change they and the city needed.

    By 1910, roughly half of Madison’s Black population lived in the E. Dayton neighborhood,

    The Miller House Historical Plaque
    William J. Toman and the Historical Marker Database

    E. Dayton's population was stable through the middle of the 20th century; however, the neighborhood started declining in the 1960s as people moved to south Madison. The city also demolished many of the original buildings. Today, only the Hill Grocery Store and the Miller's 647 house are still intact.

    Thanks in part to the legacy of the Miller's, E. Dayton Street was added to the National Register of Historic Places on March 22, 1988, as Madison’s first historically Black neighborhood.


    649-653 E DAYTON ST | Property Record. (2012, January 1). Wisconsin Historical Society. https://www.wisconsinhistory.o...

    Banks, B. (2001). Betty Banks. The People’s Stories of South Madison, 1, 4–6.

    McDonald, B. Simms, Muriel. (2018). Billy McDonald. In Settlin’: Stories of Madison’s Early African American Families (pp. 33–42). Wisconsin Historical Society Press.

    Miller House Historical Marker. (n.d.). Retrieved February 26, 2024, from

    Miller, William M., Sr. and Anna Mae Stuart · Notable Kentucky African Americans Database. (n.d.). Retrieved February 27, 2024, from

    National register of historic places continuation sheet, 10 (1988).

    Simms, M. (2018). A brief history of African American settlement in Madison. In Settlin’: Stories of Madison’s Early African America Families (pp. 5–30). Wisconsin Historical Society Press.

    Wisconsin Historical Society. Wisconsin Architecture and History Inventory. "William Miller House", Madison, Dane County, Wisconsin. 16702.

    Wisconsin Historical Society. Wisconsin Architecture and History Inventory. "Douglass Beneficial Hall", Madison, Dane County, Wisconsin. 95553

    Previous Next

    East High Girls Inc. students lead period product campaign

    March 1, 2024

    This is one example of how programs like Girls Inc. give youth the tools they need to make a difference in their community

    Girls Inc. of Greater Madison, Eastside News Stories, Community Stories

    Community Chat with Letesha: JEDI Masters

    December 12, 2023

    Letesha was joined by four local leaders to discuss justice, equity, diversity and inclusion (JEDI) work. It was an invigorating morning!

    Letesha Nelson, Community Stories, At the Center