Performance Assessments and Mathematical Thinking

In 2008, I made a major life decision to shift my career path from software development to teaching.  After leaving my teaching position in 2022, I’ve been actively searching for a new job that combines these two passions as an “instructional technologist” without much success. Part of the problem is the difficulty of communicating how I view my experiences as a teacher and programmer as two sides of the same coin – being a mathematician. As a potential method of resolving this problem, I thought I might be able to construct a “Mathematics Portfolio” that demonstrated how the skills I acquired as an educator are relevant to my qualifications as a software engineer.

There was only one problem with my plan: I’d never seen a “Mathematics Portfolio” before.  I’d seen portfolios in other disciplines related to math, including both “Teaching Portfolios” and “Programming Portfolios”, but I’d never seen a portfolio I would specifically identify as being created by a mathematician.  It’s common for mathematicians to provide a list of publications in a “Curricula Vitae”, but my working definition of “Portfolio” requires a purposeful act of curation on behalf of its creator. There’s a distinction to be made between a mathematician’s collected works and their self-evaluation of those works. A “Mathematics Portfolio” should necessarily include both the mathematics itself and a reflection on behalf of the mathematician producing it about why it is important to them.

If the word “mathematician” is defined as a “person who does mathematics”, it becomes a logical necessity to define what is meant by the word “mathematics”. This is not a trivial matter to unravel. Our definition of “mathematics” is based on a shared social understanding of what it means to “think mathematically”. Understanding the need to avoid circularities in our definitions of terms like this is one of the core components of mathematical thinking.  However, it’s very rare to actually see mathematics as an active process.  Usually this process takes place inside the mathematician’s mind and what we see is what they publish.

Perhaps it might be easier to define “mathematics” if we first define “physics”.  Human beings coexist is a shared reality that operates by a set of rules we call “physics”. We don’t get to observe those rules directly, but discover them through the scientific process. In order for us to learn how our universe works, we must first develop a model and method for testing it experimentally. We have a shared world we all interact with so our internal models are inherently similar.

In contrast, I would define mathematics as a universe-agnostic model of truth. Whereas “physics” is the study of the real universe that we collectively live in, “mathematics” is the study of all possible universes. This distinction is often described as “a priori” and “a posteriori” forms of knowledge.  However, where knowledge of physics is constructed through a separate process called science, the process of discovering mathematical knowledge is itself called mathematics.  Our definition of mathematics is inherently circular because there’s no common external model for us to compare against. Truth in mathematics is established by convincing other mathematicians that something couldn’t possibly be false.

To resolve this dilemma, I thought it might be useful to decompose mathematical thinking into a set of characteristic thought processes. While this is a topic of much debate, allow me to propose the following definitions for discussion purposes:

Mathematical thinking is a formalized thought process for arriving at truth characterized by defining terms, stating assumptions, reasoning logically, creating useful abstractions, decomposing problems, and communicating information. A person who routinely engages in mathematical thinking is referred to as a mathematician, and the collective social behavior of mathematicians is referred to as mathematics.

I’ll address each of these components in turn and discuss why I think they are important.  The goal here is simply to identify the skills that I would want my “mathematics portfolio” to demonstrate so that I can begin to organize a suitable collection of artifacts. I’ll conclude with a brief reflection about how this definition has impacted my portfolio development. 

Defining Terms

I hope that the care taken in defining “mathematics” demonstrated above provides evidence of this skill. Attempting to define mathematics alone is not sufficient to make one a mathematician, but I’d argue that not including a definition of mathematics in a mathematics portfolio would arguably disqualify one as a mathematician.  The effectiveness of mathematical thinking depends on a foundation of precise definitions. When I’m “doing math”, I mean what I say and I say what I mean.

Being a teacher has taught me that this type of formal communication is not the primary mode of thinking for most people. It’s far more practical for human beings to “code-switch” in and out of this state. After all, natural communication often involves words with double meanings that we must interpret based on the context. A word like “neighborhood” will have dramatically different meanings depending on whether I’m discussing “my house” or “a point”.  Understanding where and when precise definitions are appropriate is a quality that makes for a good mathematician.

Stating Assumptions

One of the reasons I’ve defined mathematical thinking the way I have is that it allows me to distinguish between the thought process of an individual mathematician and the collective social efforts of many. What constitutes a good definition is how well it enables further communication between two parties and coming to an agreement on the definition of terms often includes implicit assumptions about what is true in the shared reality of the communicators.  Once two people have agreed on a set of definitions it enables them to ask “what if?” questions using those terms.  

Mathematics is built on a foundation of mutually agreed upon hypothetical statements within a given society. The presently accepted de facto standard in mathematics is a specific collection of assumptions colloquially known as “ZFC” – an abbreviation for “Zermelo–Fraenkel set theory with the axiom of Choice”.  However, it’s important to note that mathematics as a whole is not limited to this specific set of rules and some interesting things happen when we choose different ones. What’s important for mathematical thinking is that we actively reflect on the assumptions we’re making and why we’re making them.

Reasoning Logically

Once we’ve established a set of well defined terms and agreed on a set of rules by which they are bound, we can start sorting out the true statements from the false ones   Given our stated assumptions and definitions, what new information can we deduce from that knowledge? This logical reasoning is arguably one of my favorite parts of math because I tend to view it like a game. Like the classic puzzle game “Pipe Dream”, mathematics is often about building a path from a starting point to the goal with a limited collection of possible moves at your disposal.

In mathematical terms, I tend to associate logical reasoning with the properties attributable to a generative grammar. Much like natural language has rules of grammar that determine what may or may not constitute a sentence, logic has an alphabet and rules of grammar that determine what may or may not constitute a proof.  What makes this game so fun, is that you may or may not be able to prove a given statement from within the system!  Mathematics can sometimes require you to step out of a system and look at the bigger picture. 

Creating Useful Abstractions

One of the most powerful aspects of mathematics lies in the way its tools effectively transfer between seemingly disconnected domains of study.  As a teacher, I often heard students often ask “when am I ever going to use this?”.  The truth of the matter is “I don’t know” and this is precisely why learning math is so exciting to me. I never know where a mathematical concept will show up, but I also wouldn’t recognize them in their natural environment if I hadn’t played with them in a “toy problem” first.  

A good example of this in action is The Matrix. In mathematics, one is generally introduced to the notion through a study of linear systems of equations.  However, the applications of matrices extend so far beyond this initial example that they are almost unrecognizable. From modeling rigid transformations in physics to modeling key changes in music, matrices provide a useful model for “connecting things” in general. In order to answer the question “What is the matrix?” one needs to both understand the properties of the mathematical object and connect them with the properties of the situation in which it is applied. Doing so essentially defines a matrix, thus giving us an abstraction for the very process by which we “step out” of a system.

Decomposing Problems

I’d like to believe that one of my strengths as a mathematician is my ability to take a seemingly intractable problem and decompose it into a collection of individually solvable subproblems. As a kid, deciding that I wanted to be a mathematician was accompanied by the simultaneous acknowledgement that I could potentially devote my entire life to working on a particular math problem and still not solve it. Learning to accept that some problems in mathematics would be forever outside my grasp allowed me to focus on learning how to break down these problems into components which I might reasonably be able to solve one at a time. Rather than immediately classifying something as impossible, I reframed the problem probabilistically over time. Given some arbitrary problem in mathematics, what is the chance I will solve it in my lifetime?

Mathematics itself is a version of the “Many-Armed Bandit” problem.  Each branch of mathematics is its own little slot-machine full of interesting problems and you have to pay-to-play this game through hours of effort.  A mathematician goes around from discipline to discipline looking for that one “jackpot theorem” – a problem that is both important and within reach. Recognizing this situation as a probability problem allows you to start to separate the problem into known strategies of “playing the machine you’ve had the best luck with” and “exploring new machines”.  This, in turn, gives you new information to work with and helps you to develop more efficient methods of addressing the overall problem. 

Communicating Information 

One of the ways my philosophy towards mathematics has been changed by teaching is through recognizing the importance of the communication process in the construction of mathematical knowledge. There’s a distinction to be drawn between “knowing something is true” and “being able to convince people something is true”. Mathematics needs to do both.  I think this distinction often gets lost while doing mathematics because the vast majority of the time the person I’m trying to convince of the truth is myself.  We all think mathematics is objective because we all want to think we’re objective self-critics.  It’s effectively a self-fulfilling prophecy.

The nature of mathematical communication is probably most salient in Analytic Geometry, where there are clearly defined and connected “symbolic” and “graphical” representations. We can correlate equations like “y=ax+b” with the shape given by “a line” as determined by plotting the points which satisfy it.  They say a picture is worth a thousand words, so it should be no surprise that a good data visualization can explain a phenomena more effectively than the numbers or narrative alone. Mathematics provides me with a vocabulary to connect the representations of thoughts between “words” and “pictures” inside my head, which is a necessary prerequisite for me to explain them to another person.

Self-Reflection

Having established a set of skills I want to showcase, I’m starting to see the difficulty in using this approach to organizing my work as intended.  More often than not, these skills have such considerable overlap that the differences between them are almost invisible.  Even this document itself, which began with explicit focus on “Defining Terms”, could equally be viewed as an act of “Decomposing Problems” when looked at in the larger context of my portfolio development.  This isn’t necessarily a problem per se, but rather indicates that the terms were chosen exceedingly well.  

My original idea was to pick out portfolio artifacts that “show off” each of these skills, but I’m now beginning to wonder if I should focus on selecting artifacts that “demonstrate growth” in each skill instead.  In my attempted definition of “mathematician”, the phrase “routinely engages in mathematical thinking” is doing a lot of heavy lifting.  People don’t develop routines without repetition, and it’s normal to stumble the first few times when learning something new. In contrast, when we see “published mathematics” most of the failures have been omitted in favor of presenting the final results in the most concise form possible.

I realized that not only have I never seen a “mathematics portfolio”, it was exceptionally rare to see a professional mathematician actually “doing mathematics”. I had a guitar instructor once tell me that when people see you play on stage they don’t see the thousands of hours you spent practicing, they only see the end result of those efforts.  Likewise, a mathematician needs to project an air of confidence in their work and showing the mistakes one made along the way might be interpreted as a sign of weakness.  However, I don’t think people learn much of anything without making mistakes and mathematics is no exception.

I concluded that in order to create a mathematics portfolio I would need to start by allowing myself space to make mistakes. It’s for this reason that I resolved to publicly work my way through a text on category theory. This may seem counter-intuitive for a portfolio, but I think that categorizing my works by the areas where I had difficulties will more effectively communicate how much I’ve grown in each of these areas. The first step to making a successful portfolio is for me to make an unsuccessful one and learn where it went wrong. It’s so logical it just might work!

Why Strive? The goals behind “Free Tutoring Tuesdays on Twitch”

At the start of the year, I decided I wanted go into business for myself as a “Freelance Mathematician” and begin offering tutoring services to the general public. I’ve been tutoring off and on for decades, but usually it was done pro bono for close friends or students enrolled in a class I was teaching. I always just did it because I enjoyed it, and anyone who knows me knows how I “light up” for a tricky math problem.

Now that I’m attempting to turn the activity into a for-profit service, I figured I would need a method of advertising to reach potential clients. It’s with that in mind that I decided to revive my Twitch Channel and put it to work. “Free Tutoring Tuesdays on Twitch” is my way of balancing my philosophy of always helping others in need with the necessity of finding some stable source of income for myself.

During one of my early streams, one of my first viewers asked if I had a specific goal in Guilty Gear: Strive (GGST). While I remarked on stream that I’ve been trying to beat the “Celestial Challenge”, I’ve been thinking that it’s something deeper than that. I’m trying to beat the Celestial Challenge with a non-binary character and on a controller specifically designed for accessibility. The reason I’m streaming this game at this time is to show how efforts by game developers to create a more inclusive environment can ultimately benefit the community at large.

The unfortunately reality is that both math classes and video game communities have historically been a hostile environment for people who are not white, male, straight, and abled. These problems are often exacerbated when competition is involved. If there’s one thing I learned while teaching, it’s that feeling safe is a necessary prerequisite for learning to take place. I want to make a place where people can come to ask questions about math or fighting games without being judged for who they are or where they are in the learning process.

I made a decision to stream GGST based on a number of factors. Some of these reasons are practical, like having a Teen rating and the ability to easily stop and start as needed, but others are more personal. I could probably do a whole post on why I’m playing Testament, but one the important points is that this is probably the first time I’ve seen a non-binary character in pop-culture that felt like a full person and not just a walking cliché. Representation matters. The steps GGST has taken to simultaneously diversify the cast, desexualize the costumes, and simplify the game mechanics, have all worked together to make it one of the most enjoyable fighting games I’ve played in years.

At the same time, becoming an “older gamer” has added a new set of obstacles to my favorite pastime. My hands just don’t work like they used to, so I was really excited when Sony unveiled their new PS Access controller:

This controller has been a game-changer for me because I can layout the buttons precisely how I want them. I wanted to be able to have a button for each finger in a position that is gentle on my wrists and shoulders. I needed to buy two Access Controllers to do this, but the important point is that I could.

Two PS Access Controllers with buttons assigned in a "hit box" configuration.
The left controller has been rotated 180°, with the following button assignments: 1: up, 2: right, 3: down, 4: left, 5: L1 (dash), Center:L2, 7:L3.
The right controller is in the default orientation with the following button assignments: 1: R1 (dust), 2: touchpad (reset), 3: R3 (faultless defense), Center: R2 (roman cancel), 5: Circle (heavy slash), 6: Triangle (slash), 7: Square (kick), 8: Cross (punch)

Sony designed originally designed this controller for people with disabilities, but those same design choices have benefits for the population at large. As a math teacher, I saw a similar effect in the classroom will specifically designed instruction. By making modifications to a lesson that address the special needs of a few, you often end up with a lesson that works better for everyone. We all benefit from improved accessibility.

In summary, my goal behind streaming GGST is to encourage the recent cultural shift towards making the fighting game community more inclusive because I think a similar shift needs to take place in education as well. I think there’s many parallels between learning how to “do math” and “play fighting games”. I was inspired in part by “educational” GGST streamers, like Hotashi, Romolla, Diaphone, and Daru I-No, who are able to look at each match as an opportunity for learning. The ability to objectively examine one’s performance is as applicable to learning math as it is to learning fighting games.

I don’t expect to be winning tournaments anytime soon. My official EVO record to date is 0-2. I might not be the best player out there, but I’ve been playing games like this for a long time and have a strong background in the mathematical principles behind them. So come watch my stream, bring your math homework, and maybe we can do some learning together!

Why you don’t see me hyping AI

There’s no ignoring the recent wave of advancement in artificial intelligence or AI. From the near photo-realistic outputs of Stable Diffusion to the pseudo-coherent text produced by ChatGPT, generative machine learning algorithms have taken the world by storm. As someone with a background in mathematics and statistics, these are without doubt fascinating advancements that I’m interested in from a technical perspective. At the same time, I have numerous concerns about these algorithms from an ethical perspective that I don’t think I’m alone in holding. That cheesy line from Spiderman about power and responsibility rings truer now than ever. To assume that we can reap the rewards of AI without accepting any risk would be a fallacy of apocalyptic proportions.

If you’re looking for a review of the literature on the hidden risks of AI, there are more intelligent people than myself that you should probably be looking for in a peer-reviewed journal. There is very little I can say about ChatGPT that hasn’t already been better articulated in Stochastic Parrots. Instead, what I’d like to offer you today is a story of a stupid teenager and his chat bot. I don’t expect it to alter the course of history or anything, but maybe it’ll provide some insights into why you should care how these technologies are being used.

I started programming in my early teens and AI was always something of a holy grail. I was especially partial to the Turing Test as an indicator of consciousness. In this test, a computer program is tasked with deceiving a human into false believing that they are engaged in a text-only conversation with another human. Turing argued that if human experimenters couldn’t distinguish between programs and people then we’d have to consider the hypothesis that machines could effectively “think”. There are arguments to be made about whether or not the Turing Test measures what it intended to, but advances in Large-Language Models have made it clear that this standard is now just a matter of time.  In fact, I’d argue that the Turing Test was “effectively passed” in the early 1980’s by a program called ELIZA developed by Joseph Weizenbaum. 

ELIZA was designed to be a sort of “virtual therapist”. A human user could talk to the computer about things that were on their mind, and ELIZA would turn their statements around to form new questions. For example, if you told ELIZA “I had a rough day at work” it might acknowledge that you’re feeling upset and inquire about it: “I’m sorry to hear that. What about your day made it rough?”. ELIZA didn’t actually know very much about the world, but it could engage a human in a fluid and convincing conversation that led the user towards self-reflection. Some users walked away from ELIZA feeling like they were engaged in dialog with a real therapist. A recent preprint from UCSD researchers indicates that ELIZA’s performance on the Turing Test is between that of GPT-3.5 and GPT-4.0.  Not too bad for a program from the 80s. 

Of course, any deep interaction would reveal the lack of “intelligence” on the other side. ELIZA couldn’t answer questions about the world. All it could do is to classify different sentences into schema and then transform them into canned responses using key tokens from the input text. Simple rules like “I feel <sad>” would get matched and transformed into “Why do you feel <sad>?”, which gave ELIZA the illusion of being a good active listener. This might sound kind of sad, but ELIZA was probably better at active listening than I am – and I knew it all too well.

As a teenager in the late 90s, we didn’t have the pervasive social media outlets we have today. There was no Facebook or Twitter for you to doom-scroll. Maybe you had a public facing Geocities or MySpace page, but that was only if you were a nerd like myself. The de facto standard for internet communication was AOL Instant Messenger (or AIM) .  Even if you weren’t subscribed to AOL’s internet access, you probably still used the stand-alone AIM application for direct messages because it was literally the only service with enough members using it to be useful. You can’t have real-time communication without a shared protocol shared by people.

The application wasn’t even that great by today’s standards. It was full of what would now be considered negligent security vulnerabilities. In early versions, you could easily kick someone offline by sending them an instant message with some malformed HTML. If someone saved their password for easy login, it was literally stored in a file as plain text and could be subsequently looked up by anyone who knew where to check. It was the wild west era of the internet.

Around the same time, I discovered a project called ALICE.  Richard Wallace had taken ELIZA’s token handling foundation and generalized it into an Artificial Intelligence Mark-up Language (or AIML).  This separated out the “code” and “persona” of the chatbot into two separate data sources.  The HTML-like syntax made it easy to customize the bot’s responses into whatever you wanted.  The application would read these source templates in and use them to craft a response to any message you gave it. 

While I’m reading article after article online trying to figure out how this stuff works, I keep getting instant messages from people. These messages are not malicious in any way, but receiving them has a tendency to “pull me out” of whatever I’m doing at the time. Sometimes I’m pulled into fun stuff through these messages, but often than not they were just an annoyance. As a teenage boy in the 90s, the vast majority of these interactions went down like this:  

sup?

WHASSSUP?

not much, u?

just chillin’

cool deal. me too.

anyways.. I got some homework to do. I’ll catch ya later!

aight. peace out!

That’s when I got the brilliant idea to fake my own AIM account.

While exploring the flaws in the AIM application, I discovered I could hijack the message queue that distributed incoming messages to the appropriate chat window.  This allowed me to parse the incoming message text and send fake keystrokes back to that window to produce a response. All I really needed to do was to invoke the chatbot as an external process to generate something to say.

I took an open source implementation of ALICE and started modifying the AIML code.  I removed any instance where the bot acknowledges itself as an artificial intelligence and instead taught it to lie that it was me. I added custom responses for the greeting I customarily used and gave it some basic knowledge about my areas of interest.  The only difficult part of this was getting the bot to handle multiple conversations at the same time, which I managed to accomplish by running separate instances of the bot for each person to message me.

I think I let the bot run for most of the week without anyone really noticing, until this one day where I got an instant message– from a girl.  Not just any girl either, but one I totally had a crush on at the time. My heart sank as I read the messages interspersed between the AI dribble.

Ryan, are you okay?

This isn’t like you.

No really, I’m worried about you.

If there’s something wrong, we can talk about it.

Please tell me what’s going on. I’m really concerned about you.

I felt sick. I immediately killed the process running the bot and seized control of AIM again. I don’t even remember what kind of bullshit excuse I made before abruptly going offline, but I’m pretty sure I didn’t have the courage to own up to the truth.  I had been “seen” through the eyes of another person in a way I hadn’t experienced before, and the worst part about it was that I didn’t like what I saw. I saw a person who lied to their closest friends in the name of science. 

I know this won’t make up for what I did, but I’m sorry.

I’ve since then learned a lot about research ethics in psychological studies.  Sometimes having deception is a necessary component to studying phenomena you’re interested in, but that is not sufficient reason to forgo the acquisition of “informed consent” from the people involved in the study.  

I think this is the reason why I’m frustrated with the current zeitgeist regarding AI. It seems like we’re rapidly falling into the trap outlined by Ian Malcom in Jurassic Park.  Some people are so preoccupied with what they could do with AI that they don’t stop to think about whether or not they should.  As a result, we’ve all become unknowing participants in an unethical research study.  While this behavior might be excusable coming from a punk teen who doesn’t know any better, this should be considered completely unacceptable coming from multi-billion dollar companies claiming to be advancing the forefront of intelligence. It’s not that “I’m scared of AI”, it’s that “I’m scared of what people will do with AI” when they acquire that power without putting in the effort to truly understand it. 

The wave of AI images coming from DALL-E and Midjourney on all my social media don’t self-identify themselves as being artificially produced. The burden of identifying them has been left to unwitting viewers and it will only become more difficult over time. While this makes for entertaining stories on Last Week Tonight, there’s a big difference between using AI to make funny pictures to share with your friends and using it to develop sophisticated social engineering methods to separate users from their passwords.  

The reality of our time is that many AI offerings are being falsely advertised as a solution for intractable problems. No image generator could possibly produce pictures of John Oliver marrying a cabbage without first being trained on a set of labeled images including the likeness of John Oliver.  Any AI image generator trained solely on ethically produced data sets, like the one from Getty, will inherently lack the capacity to reproduce the likeness of pop-culture celebrities. Either the generative AI will fail to produce the likeness of John Oliver or it was trained on a data set including his likeness without seeking prior permission.  You can’t have it both ways.  

In much the same vein, it would be impossible to ask ChatGPT to produce writing “in the style of Ryan Ruff” without it first being trained on a data set that includes extensive samples of my writing. Obviously, such samples exist because you’re reading one right now. However, the act of you reading it doesn’t change my rights as the “copyright holder”. The “Creative Commons” licenses I typically release my work under (either CC-BY or CC-BY-NC-SA depending on context) require that derivative works provide “attribution”.  Either AI will fail to reproduce my writing style or it illegally scraped my work without adhering to the preconditions.  In the event my work is stolen, I’m in no financial position to take a megacorp to court for compensation.

In discussions about AI we often neglect the fact that deception is an inherent component of how they are constructed. How effective we measure AI to be is directly linked to how effectively it deceives us. As poor of an intelligence measure as the Turing Test is, it’s still the best metric we have to evaluate these programs. When the measure of “intelligence quotient” (IQ) in humans is a well-established “pseudoscientific swindle”, how could we possibly measure intelligence in machines? If you want a computer program that separates true statements from false ones, you don’t want an “artificial intelligence” but rather an “automated theorem prover” like Lean 4. The math doesn’t lie.

I think one of the big lessons here is that the Turing Test wasn’t originally designed with machines in mind.  I still remember discovering this when I looked up the original paper in Doheny Library. The “imitation game” as originally described by Turing was primarily about gender.  It wasn’t about a machine pretending to be a human, but rather a man pretending to be a woman.  

Personally, my present hypothesis is that Turing was actively trying to appeal to “the straights” with how he described the imitation game. My incident with my AIM chatbot had taught me that there were large differences between how I interacted with “boy” and “girls”.  Conversations with “boys” were shallow and unfeeling – easily replicated by my script. Conversations with “girls”, however, were more about getting to know the other person’s perspective to determine if we’re a potentially compatible couple.  Casual conversation and romantic courtship require two entirely different strategies. Maybe the Turing Test was less about determining if machines could think and more about determining if machines could love.

Every now and then I feel overwhelmed by the flood of interaction that is constantly and perpetually produced by social media. Sometimes I wonder if my presence could be automated so that I never miss a “Happy Birthday!” or “Congratulations!” message ever again, but then I remember this story and remember that’s not really what I want.  I don’t care about “likes”. I care about building “friendships” and there’s no possible way a bot can do that on my behalf.  

Maybe I could be a better friend if I collected data on my social interactions.  At the same time, I don’t need any sophisticated statistics to tell me that I’m kind of an asshole. I’d like to think that the people I call my friends are the same people that would call bullshit on me when necessary, so I trust in them to do so. This is the difference between real human relationships and fleeting conversations with a chatbot. 

There’s nothing inherently wrong with the class of algorithms that have fallen under the “AI” umbrella, but how we use these tools matters.  Presently these bots are being marketed as a substitute for human labor but the reality of our legal system dictates that there still needs to be a human accountable for their actions.  The only viable path to survival for AI focused companies is to become “too big to fail” before they get caught for using pirated data.

I’m not going to sit here and pretend that AI won’t have its uses. Maybe AI will come up with solutions to important problems that humans would never have thought up. If every other technique for solving a problem is unreliable, there’s less harm to be caused by attempting to solve that problem through massive amounts of data. It makes sense that such statistical tools might come in handy in fields like marketing or cybersecurity where the rules of the system are ambiguously defined.  

What is clear to me is that there exist problems for which AI will never be a viable solution. GitHub’s Copilot won’t ever magically solve Turing’s Halting Problem. It’s called “undecidable” for a reason. Using ChatGPT won’t make me a better writer, nor will using DALL-E make me a better artist. There is no substitute for the process of turning my thoughts into a concrete form, and the only way to get better at those skills is to engage in them myself. Learning the internals of how AI works may have helped make me a better mathematician, but I wouldn’t expect it to solve P = NP anytime soon. 

Given my background in teaching, I was recently asked what I thought about applications of AI in education and I think it’s incredibly important that we take an abundance of precaution with its integration. This is something I’ve written about before, but I think it merits repeating that “AI needs to build a relationship of trust with all of the stakeholders in the environment”.  In our society, we depend on teachers to be “mandated reporters” for child abuse and I don’t think AI can responsibly fill this role. Without having the lived experiences of a human being, how could such an AI possibly know what symptoms are out of the ordinary?  What if it’s wrong?

Our very notion of “intelligence” is arguably shaped by the common human experience of schooling.  In my time teaching, I learned that the profession depended as much on “empathy” as it did “knowledge”.  Most of the people I’ve met who “hate math” developed this mindset in response to abusive teaching practices.  In order for AI to ever succeed in replicating good teaching, it needs to learn “how to love” in addition to “how to think” and I don’t think it ever can.  

Even my use of the word “learn” here seems inappropriate. AI doesn’t technically “learn” anything in a conventional sense, it just “statistically minimizes an objective cost function”.  Seeing as “love” doesn’t have an objectively quantified function, it therefore it’s impossible to replicate using the existing methods of machine learning.  Even if a machine were capable of expressing love, the act of replacing a human being’s job with such an AI would go against the very definition of what it means to love.

As with any new technology, AI can be used to make our lives better or worse depending on how it’s used. Let’s not lose sight of “why” we’re constructed these systems in the first place: to improve the quality of human life. Building these programs comes with a very real energy cost in a world where humans are already consuming natural resources at an unsustainable rate. If the expected pay-off for these AI systems is less than the expected environmental costs then the only responsible course of action is to not build it.  Anyone who fails to mention these costs when it comes to their AI product should be treated as nothing short of a charlatan.

I can’t shake the feeling that we’re in the midst of an “AI hype bubble” and it’s only a matter of time before it bursts. I can’t tell you not to use AI, especially if your job depends on it, but as your friend, I feel it’s important for me to speak up about the risks associated with it. 

True friends know when to call bullshit.

Find me in the Fediverse

TL;DR? Pick a server to join. Follow me. Share cat pics.

I’ve learned to accept that my blog posts tend to come in waves, but just because I haven’t been blogging recently doesn’t mean I haven’t been writing. In fact, I’ve been actually posting rather consistently for the past couple months over on Mathstodon. More specifically, I’ve been engaging in “Category Theory Caturday” where I share cat pics and math on a weekly basis. If you follow me on other platforms, you may have been missing some quality cat content such as this:

A meme-like format pairing photos of a large brown tabby with text. The first picture depicts the cat staring forward solemnly with the text "When the math book includes exercises.". The second picture depicts the cat staring up and off camera with motion blur around the head aside text that reads "When the math book includes parenthetical side questions.
Sample “Category Theory Caturday” photo

Now, I hate to disappoint you if you’re following me on other platforms but “Category Theory Caturday” will most likely stay exclusive to Mathstodon for the time being. I don’t want you to think you’re missing out on it, but I also want you to have a reason to join the Fediverse.

The Fediverse is not really about an app, but an idea. It’s the idea that the value of social media lies in developing a relationship of trust between participants. It’s the idea that the people who produce the content should control that content. It’s the idea that the people or hash tags you explicitly choose to follow should determine the content that appears on your feed.

The Fediverse is not Facebook or Twitter and this is simultaneously both a blessing and a curse. Social media didn’t have to be about selling your eye-time to advertisers in a never-ending quest for more click-throughs, but that is ultimately what these giant websites have sculpted it into in an effort to keep the business model sustainable. In contrast, the Fediverse solves this problem through decentralizing the social network using a common standard called ActivityPub. It’s like having a bunch of mini-Facebooks and mini-Twitters that have all mutually agreed to cross-share content between their communities. The “app” I use to connect to the Fediverse is called an open source web application called Mastodon which enables anyone to easily host their own server. However, the downside of being in a system of inter-connected communities is that you need a community to start with that can absorb the operational costs that hosting a server necessitates.

This is the hardest part about asking you to join me in the Fediverse is that in order to “join me using Mastodon”, you must first choose a home server. This is difficult because you don’t really know how this will impact your experience until you do it. Each server has its own vibe. You’re just going to have to go with your gut feeling and take comfort in knowing that you can always migrate later. Today I’d like to share my experience so that you might have a better idea of what to look for when making that decision.

I was fortunate to be part of a existing community of math teachers that migrated from (the site formerly known as) Twitter to Mastodon en masse. I’ve been on Twitter for years, so as people started posting their new handles in my feed I started adding them. It’s not even an understatement when I say that the methods of that migration were actively hindered by Twitter. The decision to join the server I did was made easy when I saw that the admin, Christian Lawson-Perfect, had added support for mathematical expressions through LaTeX. If that that doesn’t mean anything to you, that’s okay; it’s just a very specific technology that enables mathematicians to better communicate with one another. That’s the power that a properly selected home server can have on your experience. Once I chose a server, it was a fairly easy process to log in to it through the official Mastodon app on my phone. The only downside to the official app is that it doesn’t have LaTeX support (yet).

After joining a server, there were some settings that I changed that greatly improved my experience. The first was to enable dark mode, which makes it way easier on my eyes. The second was to “Enable Advanced Web Interface”, which allows you to set up a multi-column view of incoming “toots” in real time. These two changes make the Mastodon interface look and feel more like the “Tweetdeck” view I had grow accustomed to over the years. I also turned on the “Always show media” option but choose not to “expand posts content warnings”. The people in my network have been generally good about using content warnings where appropriate.

One of the advantages to being having my home server being “Mathstodon.xzy” is that I get to see my server’s “local timeline”. Each time someone from the server posts something, it shows up here. It’s really fun to see the juxtaposition of posts containing complex mathematic research with deeply personal glimpses into the lives behind it. There’s also a “federated timeline” that combines my server’s feed with a combination of every connected server, but that feed is way too fast and chaotic even for me.

One of the downsides of the platform is that the user base hasn’t quite reached critical mass yet and it can feel a bit lonely when you only have a limited number of people you’re following. I found that one of the easiest solutions was to “follow hashtags” in addition to people. This takes that federated timeline and filters it down to search terms you’ve explicitly added. I might regret following #CatsOfMastodon at a later point in time, but right now there are so few posts in general that I get an acceptable quantity of cat pics in my feed. The community driven nature of the moderation system seems to be keeping the majority of spammers away for now.

It’s important to understand that the Fediverse has something of a “do it yourself” culture behind it. I’m not going to lie to you and say that it didn’t take effort to construct a feed of interesting content, but only that what I produced was worth the effort. When I log into other platforms now, I find myself scrolling through an endless stream of the content “they want me to see”. Being able to choose the content that “I want to see” and get it is so empowering that I don’t think I can go back.

That’s not to say the platform doesn’t have problems. It does. There are going to be problems with any sufficiently large social media app. The code needs to run on a physical server somewhere in the world and is subject to laws thereof. There is a risk that the government will seize your host’s server if they suspect illegal activity. Your messages in Mastodon are not end-to-end encrypted and could theoretically be read by a server administrator. For these reasons, it’s popular for Americans like me to choose a server in the European Union where there’s better data privacy laws. You’ll also need to mitigate your own risks by not posting any sensitive information to begin with.

With that being said, if you’re looking for a social media outlet for a larger organization, you also have the power to start your own server! Perhaps your email server is a good analogy for a Mastodon instance. Sure, you can get free email from a search engine giant like Google or Yahoo, but there are hidden costs associated with it because the organization needs to “keep the lights on” somehow. Many servers, like mine, are supported through donations. Consider helping your local admin out if you’re able to.

If you’re running your own business, you can probably afford to buy a domain “YourCompany.com” so you can have a branded email address like “you@YourCompany.com” that you have complete control over. Rather than having to rely on brand recognition of an icon that could on changed spontaneously on the whims of an idiot, you can post your updates to a Mastodon instance that you control down to the bare-metal. If you don’t feel a need to go quite that far, you can still use your control of “YourCompany.com” to verify your identity on whatever server you do decide to grace with your presence. Nobody can threaten take your verified checkmark from you either!

Anyway, I hope that this post has been informative and gives you the courage to try it out if you were on the fence about it. Here’s that link to join again if you need it. I look forward to seeing you when you get there and will try to continue keeping a steady supply of math and cat pics!

Coursera Speed Trials

On May 8th of 2019, I enrolled in the Deep Learning Specialization on Coursera. Coursera had offered a “30-day free trial” on the three-month long course program and I took that offer as a personal challenge. By June 5th I had completed my first certification without spending a dime.

It’s easy to see why hiring managers might be skeptical of micro-certifications like this one. I mean, if I was able to complete the in program in 28 days, how hard could it really be? At the same time, I also know that I was only able to accomplish this feat because I had substantial background knowledge in linear algebra and computer programming to begin with.

The reality of the situation is that I didn’t take the course for the certificate, but solely because it sounded fun. And it was. It reminded me how much I enjoy the process of learning and helped me to find joy in my teaching again. I’d do it over again in a heartbeat.

So I did.

On February 14th 2020, I started the “IBM Data Science Professional Certificate”. I completed the five-month course sequence by April 2nd.

On September 29th, 2021, I started the “IBM Full Stack Developer Professional Certificate”. I completed the four-month course sequence by October 31st.

I know quite well that learning is not a race. Finishing these courses early is not necessarily a valid indicator of skill. However, I do think the manner in which I completed them provides insight into the type of person I am and my attitude towards self-improvement.

For better or for worse, I’m a thrill-seeking-binge-learner. When I get curious about a topic, I just dive right in. The more difficult the task, the more I want to do it. I’m the type of learner who desires challenge so fiercely that I would place additional time constraints on myself as a handicap.

Reflection: Deep RL with Hugging Face

Certificate of Excellence for the Hugging Face Deep Reinforcement Learning Course issued to Ryan Ruff on 3/8/2023
Look! I did a thing!

Back in January, I learned about the Hugging Face Deep Reinforcement Learning Course from Thomas Simonini and decided it would be fun to participate. Now that I’ve finished the course, I thought it would be a good idea to reflect back on what I learned from the experience and offer some feedback from an educational perspective. For brevity’s sake, here’s a high level summary of what I found to be the pros and cons:

Pros:

  • Using Google Colab made it very quick and easy to get started.
  • The sequence of problems was very well thought out and flows nicely from one to the next. I felt like each unit does an excellent job of setting up a need that gets addressed by the following unit.
  • It finds a nice balance between the mathematical and natural language descriptions of the discussed algorithms.
  • I enjoyed the use of video games as a training environment. Watching the animation playback provided helpful feedback on agent progress and kept the lessons engaging.
  • I developed an appreciation for how gym environments and wrappers could be used to standardize my code for training experiments.
  • I feel I now have a much better understanding of how the Hugging Face Hub might be useful to me as a developer.

Cons:

  • The usage limits on Google’s Colab eventually became a hinderance. It might be difficult to pass some of these lessons if that’s your only resource and some of the units suffer from nightmarish dependencies if you try to install locally.
  • I really disliked the use of Google Drive links in lessons, particularly when they contained binaries. I’d feel a lot safer about trusting this content if it came from a “huggingface.co” source.
  • Some of the later units felt little unpolished in comparison to the early ones. It was a little frustrating to spend increasing time debugging issues that turned out to be mere typos (“SoccerTows”) while also having drastically less scaffolding to work with.
  • Accessibility of these resources seemed very limited. Some of the text in images was difficult to read and lacking alt text. Some of the video content would benefit from a larger font and slower narration.
  • While training bots for Atari games was certainly fun, the lax attitude towards attribution is concerning from an ethical and legal standpoint.

Overall I had an enjoyable time going through the course. I found myself looking forward to each new unit and cheering on my model as I habitually refreshed the leaderboards. I did not, however, join the Hugging Face community on Discord, so I can’t comment on what the social elements of the course were like. Nothing personal, I just dislike Discord.

It’s probably also important to note that I came into the course with some prior experience with both Python and Deep Learning already. For me, I felt this course made a nice follow-up to Andrew Ng’s Deep Learning Specialization on Coursera in terms of content. While it might be possible to make it through the course without having used PyTorch or TensorFlow before, I feel like you’ll probably want to at least have a conceptual understanding of how neural networks work and some basic familiarity with Python package management before starting the course.

My favorite lesson in the course was the “Pixelcopter” problem in Unit 4. This was the first time in the course where hitting the cut-off score seemed like a real accomplishment. I probably spent more time here than in the rest of the course combined but felt like it was this productive struggle that enabled me to learn some important lessons as a result. Up until this point I felt like I had just been running somebody else’s scripts. Here, it felt like my choice of hyperparameters made a huge difference in the outcome.

Part of the problem with choosing hyperparameters for this Pixelcopter was trying to keep the training time under the time constraints of running in Google Colab. If the training time was too short the bot wouldn’t produce a viable strategy, and if the training time was too long then Colab would timeout. At this point, I went back to the bonus unit on Optuna to manage my hyperparameter optimization. I was able to get a model that produced what I though was a high score, but the variance was so high that I didn’t quite the cut-off score.

Eventually I got so frustrated with the situation that I set up a local Jupyter-Lab server on an old laptop so I could train unattended for longer periods of time. However, this came with its own set of problems because I mistakenly tried to install more recent versions of the modules. Apparently the “gym” module had become “gymnasium” and undergone changes that made it incompatible with the sample code. In an effort to keep things simple, I rolled gym back to an earlier version so I could concentrate on what the existing code was doing.

Once I got my local development environment going, I let Optuna run a bunch of tests with different hyperparameters. This gave me some insight into how sensitive these variables really were. Some bots would never find a strategy that worked. Other bots would find a strategy that seems to work at first, then something would go wrong in a training episode and the performance would start dropping instead. With this in mind, I decided to add an extra layer to my model and started looking more closely at the videos produced by my agents.

What I noticed from the videos was that some bots would attempt to take the “safe” route and some bots would take the “risky” route. The ones that take the safe route tended to do well in early parts of the episode, but started to crash once the blocks speed up enough. The ones that try to take the risky route do way better on later stages, but the early crashes result in unpredictable performance overall.

In an effort to stabilize my agent’s performance, l started playing around with using different environment wrappers. The “Time-Aware” Observation Wrapper seemed to help a little, but I ran into a problems with gym again when I attempted to implement a “Frame Stack”. Apparently the there was a bug in the specific version I had rolled back to, and explicitly pinning my gym version to 0.21 resolved the issue. With a flattened multi-frame time-aware observation the bot was able to come up with a more viable strategy.

Video showing my Pixelcopter bot’s growth with more input data

What really drove this lesson home was that I felt it set up a real need for the actor-critic methods in Unit 6. I knew precisely what was meant by “significant variance in policy gradient estimation”. I also learned in Unit 6 that the reason the Normalization wrapper I tried before wasn’t working was that I didn’t know I had to load the weights into my evaluation environment. All of these small elements came together at the same time. My extensive trial and error time with Pixelcopter in Unit 4 had show me precisely why that approach would be insufficient for the robotics applications in Unit 6. I felt like understanding this need really solidified the driving ideas behind the actor-critic model.

I also thoroughly enjoyed the “SoccerTwos” problem in Unit 7. However, the part where I had to download the Unity binaries was very discomforting. Not only was the link hosted on “drive.google.com”, but the files inside the zip folder were misnamed “SoccerTows” instead of “SoccerTwos“. It looks like this issue may have been corrected since then, but I won’t deny it caused a moment of panic when I couldn’t find the model I’d been training because it wound up in a slightly different location that I expected. I feel like Hugging Face should have the resources to host these files on their own, and the fact there were typos in the filenames makes me wonder if enough attention is being paid to potential supply chain vulnerabilities.

My least favorite unit had to be Unit 8 Part 1. I felt like I was being expected to recreate the Mona Lisa after simply watching a video of someone painting it. I didn’t really feel like I was learning anything except how to copy and paste code into the right locations. And this might be a sign of my age, but it was extremely frustrating to not be able to clearly read the code in the video. Some of the commands run in the console are only on screen for a second and it’s not always clear where the cursor is at. The information may be good, but the presentation leaves much to be desired. As a suggestion to the authors, I’d consider maybe splitting this content up and showing how to set up a good development environment earlier in the course so you can focus more on the PPO details here.

While fun examples, I also felt a little uneasy with the way the Atari games were included in this course. Specifically, the course presents Space Invaders in a manner that seems to attribute it to Atari when it was technically made by Taito. I feel like this is more a complaint for OpenAI as the primary maintainer of gym than it is toward Hugging Face, but I got the distinct impression that these Atari games the RL zoo are technically being pirated. After finding this arxiv paper on the subject, it looks like OpenAI erroneously assumed that the liberal license to the code in this paper gave them justification to use the Atari ROMs as benchmarks for large scale development. Given that these ROMs are being used to derive new commercial products, what might have been “fair use” by the original paper is now potentially copyright infringement on a massive scale. I strongly believe the developers of Space Invaders deserve to both be cited and paid for their work if its going to be used by AI companies in this way.

In conclusion, I think completing this course gave me a better understanding of what Hugging Face is attempting to build. The course taught me the struggle of providing reproduceable machine learning experiments and demonstrated the need to have a standardized process for sharing pre-trained models. This free course is a great introduction to these resources. At the same time, the course also drew my attention to the ways this hub might be misused as well. I think I would feel more comfortable with using models from the Hugging Face Hub if I knew that the models hosted there were sourced from ethically collected data. A good starting point might be to add a clearly identified “code license” and “data license” on project home pages. While Hugging Face says this should be included in the model’s README, a lot of the projects I saw on the hub didn’t readily include this information. I sincerely hope Hugging Face will take appropriate efforts to enforce a level of community standards that prevent it from turning into into “the wild west” of AI.

In any event, thank you to Thomas Simonini and Hugging Face for putting this course together. I really did have a fun time and learned a good deal in the process!

The models I built during this course can be found through my Hugging Face profile here.

On Being Purple

Do you ever wonder if we do a disservice to children by asking them to name a favorite color too early in life? With such limited life experience, by what criteria does a child even decide? It seems like innocuous small talk on the surface, but once you’ve named your favorite color there’s a tendency to commit to that decision for the long haul. It’s incredibly unlikely that I’ll wake up tomorrow and spontaneously declare that now my favorite color is blue. That doesn’t imply preferences can’t change over time either.

Maybe it’s the innocence intrinsic to the childhood experience that gives our answer such impact. When I asked my young nephew what his favorite color was, I could see clear physical indicators of the thought being put into his response. His eyes rolled up and back, he moved his hand to his slightly elevated chin, then replied “purple!” with an unmistakable air of confidence. The certainty of his assertion in juxtaposition to his age is so awe inspiring that I’m almost jealous. We know the kids are right. The fact that I feel a need to protect him from the prejudices of society makes me wonder just how much of my preferred palette is a product of the environment where I grew up.

The reality is that my favorite color was undeniably influenced through social interactions. I distinctly remember a time when would tell people that my favorite color was black. This caused problems with other kids my age. “Black’s not a color! It’s the absence of color!” was so frequent a response that the very question of my favorite color would make my blood boil. It would make me see red. At some point I grew so tired of always having to argue with people about the definitions of color and hue that I eventually gave up the fight and started saying red instead. It worked rather effectively. People tend to associate red with anger and cede you more space as result. This can be quite rewarding when you’re as introverted as I am. It’s no surprise that the choice stuck, but something about this never sat right with me.

Part of me feels guilty that I didn’t defend my choice of black more strongly than I did. Every computer I’ve ever worked with has treated black as a valid color. Looking back, there’s an obvious explanation for this anti-black sentiment that I was too young to understand at the time. No one notices the contrast of white on white. If I would have claimed white as my favorite color, would it have been subjected to the same level of scrutiny for being “not a color but all colors”? I’m not really sure. The people that seem to most enjoy the fog are the same people that get offended when light passes through a prism.

Another part of me feels slightly hypocritical for simultaneously loving red and hating pink when I know the two colors are essentially the same hue. There was a very prevalent stigma attached to boys who liked pink. Much like the people who attacked my choice of black as a non-color, I engaged in similar form of mental gymnastics to assert that my favorite color was red was not pink. Perhaps it’s not really a question of “if” I was pressured into my color choice by social factors, but to what degree I tried to be what other people think of when they see me. There’s no denying that I preferred blood over bubblegum in my shades of red.

I’ve never been particularly good at understanding how people see me. There’s this shared connection people tend have between colors and feelings but sometimes I’m not sure I experience things quite the same way. If you lead me down a rainbow hallway I’ll choose the red door, but I will simultaneously see that red door and want it painted black. It’s hard not compare the way I tend to suppress my feelings with the way I avoid bright colors, but I think saturation and intensity both fall short of the metaphor I’m looking for here. Black is not necessarily the absence of feeling. Black is the shade on a hot summer day that provides you with reprieve from the relentless sun. Black is a warm trench-coat on a cold winter’s morning. Sometimes I need that armor which only the void can provide.

To make this analogy between colors and emotions work, perhaps it would make sense to add the concept of an alpha channel. In graphics programming it’s common to augment our definition of color from red, green, and blue to include opacity so we can express an image in layers. Emotions work like this also. Sometimes an emotion is relatively translucent and I see through it well enough to go on with my day. Other times an emotion is so opaque that I literally can’t see anything else. Grief is often the primary culprit of this. Grief stacks on layer after layer of blue on blue, heartache on heartache, until your vision disappears completely.

I already had a hard time distinguishing feelings, so learning how to recognize the layers of multiple emotions together was a difficult process. I could normally manage my rage on its own, but this phenomena of simultaneously being sad and angry was a burden I wasn’t prepared for. I felt lost in this purple haze, not knowing if I’m coming up or down. It used to be that red made me happy by pulling me out of my blues, but now the misery just follows with me. It’s like this painful bruise under my skin that I just have to bear with until it heals. Perhaps this is how purple came to be associated with courage.

Perhaps somewhere deep down I’m afraid of what purple represents. It’s so much easier for me to logically separate purple into red and blue than emotionally engage in the combination. When you find your pleasure in clouds of bright red cotton candy, it’s trivial to direct your hostility towards the cold blue steel it’s locked behind. Having a clearly defined prey to hunt makes life simpler for the predator. It’s much harder to accept that I’m drawn to the prowl because the thirst for blood distracts me from the river of tears I’m floating down. The reality is that I was raised in a world which portrays feelings as an impediment to survival and purple got caught in the crossfire.

I find it interesting how if you filter out the red and blue from white light you’re left with green. It seems like an appropriate metaphor for the unsatisfied hunger I feel. When you’re surrounded by mountains of purple, the grass is always greener on the other side. Feeling nothing at all would be preferable to feeling pain, but for every step forward I take I wind up sleepwalking back again. It makes me wonder if my efforts at suppressing feelings are ultimately futile. Maybe I need to learn how to scout my feelings from a higher altitude so I can figure out which bridges actually lead to verdant pastures and which ones I’ve already reduced to embers.

It’s here on this boundary between red and green that I’m starting to find hope again. In this stop and go world, we tend to think of red and green as opposites because that’s how our eyes work. However, it’s important to remember that these colors can be combined. When you mix red and green paint together it turns a disgusting greyish-brown, but you add red and green light together it produces a brilliant shade of yellow. A simple change in perspective can have huge consequences. It used to be that I associated yellow with fear, but now I’m starting to see it more optimistically as an opportunity for growth. Embedded in this line between red and green is a whole spectrum of yellows from the shade of fertilizer to shining sun. When the traffic signal turns yellow, do you speed up or slow down? Sometimes we need to do something that seems scary at the time in order for it to turn into something beautiful later.

Back when I was a teenager, I owned a pair of sunglasses with amber lenses and noticed something interesting happen when I’d take them off after wearing them for a while. Everything turned blue. In much the same way our eyes treats red and green as opposites, they also treat blue and yellow. When everything you see is displaying shades of gold naturally, your brain gets used to seeing the world like that and begins to filter it out. It seems fitting that yellow and blue have this relation. Trying to see hope in everything is so exhausting that I wind up seeing only the sorrow.

I think what makes these feelings difficult is my lack of control over them. They come down on me like rain. Part of me knows that both the sun and rain are necessary for growth. That’s the only way the roses bloom. The other part of me is scared to embrace what I can’t control. Bruce Lee taught me that there was power in being like water, but sometimes when it rains it floods. In times like that it’s Brandon’s line from The Crow that keeps me moving forward. I tell myself “it can’t rain all the time” as I try desperately to make my peace with the tears.

The first book I that I recall reading by a Black author was Alice Walker’s The Color Purple, so I revisited the film recently in preparation for this post. To say I didn’t understand it at the time would be a massive understatement. Looking back, part of me is kind of glad that I didn’t get it. I was so privileged that I didn’t even have a frame of reference for that level of suffering. I didn’t know what it was like to stand in the purple rain and be resigned to watch as destruction falls from the sky on everything you care about. I couldn’t imagine the courage it takes to feel that kind of pain and still be capable of laughter and compassion. In some ways it was better for me to not know these things in the same way I do now. Some feelings are best left unfelt.

In an effort to show solidarity with my nephew, I made it a point to wear my purple shirt around him. Note that I say “my” instead of “a” because at the time of writing this I literally only have the one. I’m the type to primarily dress myself in blue, red, black and grey. Heck, I even bought myself a grey guitar to play. Yet, my lone purple shirt represents something special to me. I only own it because Dr. Val Brown and a compassionate group of educators decided to #ClearTheAir by openly talking about racism on Twitter. It features the following quote from Dr. Martin Luther King Jr.:

“…persistent trying, perpetual experimentation, persevering togetherness. Like life, racial understanding is not something we find but something that we must create.”

As I’ve started to wear it more, I’m finding that I’ve become increasingly more comfortable seeing myself in shades of lavender and indigo. Maybe I look good in purple. Maybe there’s some alternate universe where my favorite color is violet or magenta– a place where I felt truly free to dream in color. Yet I’m so used to dreaming in ones and zeros that I have a hard time even envisioning that in my head. Somehow my brain can imagine that it sounds something like music though– a harmonious cacophony.

It’s easy for me to imagine dreams of blue. It’s easy for me to imagine dreams of red. But dreams of purple still feel elusive. The color itself feels like an illusion. Colors are to light what pitch is to sound. Red light has a low frequency and blue has a high frequency. Purple is a frequency of light that’s so much higher than blue that it starts to appear red again. Because the range of our vision is limited, we perceive the top and bottom of the color spectrum as forming a loop. Perhaps dreaming in purple is like dropping or raising your voice an octave to harmonize with a signer that’s outside your range– two dreams coming together as one.

I know better than to assume I can just snap my fingers and be in a different world. At the same time, I don’t want to give up hope for a better one either. It’s not within my power to remove all possible sources of pain from the world, but maybe there are steps I can take to share the load. Just because red is my favorite color doesn’t mean I can’t dabble in adjacent colors. There’s plenty of passion to be found between grapes and gold. Maybe allowing myself to space to express myself through purple can be an act of resistance that provides a crack in the clouds for hope to shine through.

Maybe someone out there needs to see that it’s okay to be purple.

Maybe that someone is me.

[I’d like to thank the following sources for the vibe that helped me get through this: Robert DeLong, K. Flay, WALK THE MOON, Lola Blanc, Counting Crows, Elliot Lee, The Rolling Stones, Bobby Vinton, Jimi Hendrix, Oingo Boingo, Pink Floyd, Coldplay, 311, grandson, Jessie Reyez, Prince, and The Art of Noise. Thanks for reminding me what it means to feel human.]

Converting YouTube Music “likes” to Spotify “favorites” Using Python

Introduction

I’m planning to do something a little different with this post. It’s probably more code heavy than my usual writing and more wordy than my usual coding. I wanted to share it because I think it’s simultaneously an interesting story and potentially useful script. My hope is to provide a gentle introduction to Application Programming Interfaces (or APIs): what they are, how to use them, and why having open access to them is so important in today’s society. Some familiarity with Python will be helpful when it comes to understanding the code snippets, but feel free to skip over those sections if you’re not feeling it.

Furthermore, I’d also caution in advance that this post contains a brief discussion of self-harm related content and mature language. It’s okay to set boundaries for yourself and skip to the code if that topic is sensitive.

While the narrative is my own, the code here is based on code samples from Google and Spotify licensed under the Apache 2.0 License.

Google’s quick start guide is available at: https://developers.google.com/youtube/v3/quickstart/python

Spotify’s quick start guide is available at: https://developer.spotify.com/documentation/web-api/quick-start/

You may obtain a copy of the Apache 2.0 License at: http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an “AS IS” BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

The Python code that follows is designed to be run in Google Colab. Hypothetically, you should be able to open this code in your own notebook by using the following link: https://colab.research.google.com/github/rruff82/misc/blob/main/YTM2Spotify_clean.ipynb

The reason I’m using Colab for this is that it makes it extremely easy to authenticate to your Google account. Please understand that doing things the easy way carries inherent risks, and you should be mindful of any code you run in this manner.

If you’ve never used an interactive notebook before, it’s broken down into “cells” which can be either text or code. I’m going to talk you through all the steps I went through to migrate my likes and describe what I was thinking at each stage of the process. You’ll need to run each code cell one at a time as you follow along. You can do this directly in your web browser by selecting a code cell and either clicking the “play” icon on the left or by using CTRL+Enter on your keyboard.

Background

This story begins back in 2014 when Google did a collabaration with Marvel to release the Guardians of the Galaxy soundtrack for free on Google Play Music. Total marketing campaign success. It got me using the app more frequently, and the more I used it, the more I loved it! Before long I was buying an album a month and it seemed totally reasonable to pay for a subcription to the service.

The performance of the Play Music app was top notch. I especially appreciated the option for caching songs while on WiFi so I could listen to my favorites without eating through my cell phone’s data plan. It also gave me the ability to upload some of the more obscure albums in my collection and easily integrate them into playlists with subscription content. As I started liking songs on the platform, the algorithm started to get really good at providing me with new music that I actually liked. Furthermore, the integration with Google’s voice activated assistant was perfect for controlling my music mix while driving.

Unfortunately, Google terminated the Play Music service in 2020 and replaced it with YouTube Music. They migrated over my library well and my Play Music subscription had been upgraded to suppress ads on YouTube, but as a music service it wasn’t really meeting my needs. Songs started cutting out when my phone changed networks. The voice commands I relied on while driving stopped working. I’m also pretty sure the web app had a memory leak which would cause it to crash if I listened to a single station for too long. Everything that I loved about “Play Music” gradually slipped away.

The nail in the coffin came in 2022 when Google started rolling out content filters on YouTube that flagged videos for depictions of “self-harm”. I can understand that this is a good idea in theory, but it essentially made YouTube Music inoperable for me because there was no way to bypass this warning within the YouTube Music app. Rage Against the Machine’s entire debut album became inaccessible on the platform overnight — presumably because of the cover art’s depiction of self-immolation. What was most frustrating was that this ban was predominantly affecting playlists I had explicitly made to cope with the grief of having lost close family members to suicide. When you’ve relied on certain playlists of music to help you process painful emotions for years, having that safety net abruptly taken away from you by a third-party vendor feels pretty fuckin’ horrible. And then they had the balls to try to raise the price of their shitty service too? I felt so betrayed.

I don’t think Google understood the role music really played in my life. They knew the music was a commodity that I was willing to pay for, but they didn’t know the extent to which music was the cornerstone of my emotional foundation. Heck, I don’t think I understood it fully either. Other people talk about feeling as originating from the heart, but I’ve started to wonder if I’m different. I feel like my feelings are closely tied to the physical sensations I associate with music. It’s in my fingers when I feel the vibration of the strings against my fingers as I press down up on the frets. I feel it in my lungs when I try to belt out that falsetto. I can feel it resonate through my whole body when I stand to close to the bass amp. Music is more than just a background soundtrack in my life; the rock opera is a mirror to my soul.

For better or worse, Google had forced me into paying closer attention to how much of my music library was connected to suicidal themes. It was disturbing see some corners of my music collection getting banned while also seeing obvious references go unnoticed by the algorithm. The lyrics to “Pardon Me” by Incubus describe the same event as shown in Rage Against the Machine’s album cover but somehow wasn’t affected by the filter. Who gives a damn what the album cover looks like if I’m listening to it in my car? Trying to protect me from unanticipated self-harm related imagery in music is a impossible task when I grew up listening to artists like Kurt Cobain, Chris Cornell, and Chester Bennington. The line between the music that heals and the music that hurts is one that that only I can draw.

The problem with migrating to a new music service lies in training it to understand which music I want and what I don’t. Over the 8 years I used Google’s music services I had accumulated over 2200 “likes” and built dozens of playlists. This music profile has an emotional value that made it difficult to leave the platform. Cory Doctorow recently wrote about the “enshittification” of social media platforms, and I couldn’t help but feel that’s essentially what happened to Play Music. I was locked in by my data and needed a way out.

Having made a decision to leave, signed up for Spotify and found a service called TuneMyMusic that imported some of my key playlists. However, there was a 500 song limit on what they’d transfer for free and the playlists I did import weren’t as accurate as a would have liked. There was also the problem of it playing a lot of music that I didn’t like on radio because my library of favorites was still empty. Hearing Creed in my alt rock radio is enough to make me want to break shit. That’s when I decided I would use the Spotify API to just move my Youtube Music “likes” over myself.

There’s something empowering about being able to do this. The continual enshittification of Twitter serves as a constant reminder of the powerful impact of API access. Despite all its flaws, Google had graciously provided me with all the tools I’d need to exit it’s own platform — provided I was willing to put in some effort. I respect that and it’s comforting to know I can always reverse this process later if I so choose.

At the same time, this is probably not something the “average person” would be able to do on their own. But I think they could do it with assistance! I wrote this code primarily for myself, but hopefully this will provide a detailed enough example for others to build on.

I don’t think of APIs as a difficult concept to understand. They’re basically just a computer’s version of a contract for certain services. The difficulty lies mostly in knowing of their existence in the first place. You have to know who provides the service and what they allow you to do with it. Without some idea of what’s possible using an API, you’d have no idea what you could use them for!

Code

First, we’ll need some credentials to use the APIs. These credentials contain sensitive information that proves we’re supposed to have access to the APIs we want to use. It’s good practice to keep these secrets separate from your code, so I’m storing them in Google Drive because it’s convenient.

PLEASE DON’T SHARE THESE FILES WITH ANYONE!

For Youtube Music, you’ll need to first open up the Google Cloud Console:

Go to “Credentials” and select “Create OAuth client ID”. Configure the consent screen then create and download OAuth client secret. Upload this file to google drive as “creds_google.json”.

For Spotify, you’ll need to go to the Developer Dashboard:

Select the option to create a new application. Where it asks you for a “redirect URI”, add “https://localhost:8888/callback”. We won’t actually use this since we’re not building a full web application, but we need to put in something here so we’re choosing something that would be useful in a debugging environment later.

Once you’ve created the credentials, take note of the “client_id” and “client_secret”. You’ll also need your Spotify username which you can find on the account overview page:

Using your favorite text editor, create a new text file containing the information above using the following JSON format:

{
  "client_id":"REPLACE WITH CLIENT ID",
  "client_secret":"REPLACE WITH SECRET",
  "user_id":"REPLACE WITH SPOTIFY USERNAME"
}

Save and upload to Google Drive as “creds_spotify.json”.

Next, we’re going to install a Python module to simplify access to the Spotify API. The following command will tell Colab to install the latest “Spotipy” package from the PyPI repository. We do this first because we might need to restart the interpreter afterwards.

!pip install spotipy

Now that the “Spotipy” package is installed, we’ll import all the modules we’ll need for the rest of the script. Typically I import them as I produce my code but collect them at the top. This makes it easy to check that all my prerequisites are accounted for before I get too deep into my code.

# The first section are modules we'll need to read our credentials files in
import json
import os
import os.path

# The following are modules for accessing the Spotify API using the package
# we just installed.
import spotipy
import spotipy.util as util
from spotipy.oauth2 import SpotifyOAuth
from spotipy.oauth2 import SpotifyClientCredentials

#  We'll need "sleep" so that I can limit the rate of my Spotify API calls
from time import sleep

# The following are modules that we'll need to authenticate to the Youtube API
import google_auth_oauthlib.flow
import googleapiclient.discovery
import googleapiclient.errors

from googleapiclient.discovery import build
from google_auth_oauthlib.flow import InstalledAppFlow
from google.auth.transport.requests import Request
from google.oauth2.credentials import Credentials

from google.colab import drive

Since we stored our credentials in Google Drive, we’re going need to give ourself access to it from this notebook. The following line will tell the interpreter to treat our Google Drive as a local folder. Please exercise caution when doing this as we’re technically granting more access than we actually need.

drive.mount('/content/drive',readonly=True)

Now that we have access to our credentials files, we’ll need to authorize the application to read our playlists from Youtube. The following code will prompt you to log in with your Google account and warn you that you are providing your information to an application that’s under development. This is okay because we created the app. Copy and paste the code from the confirmation screen into the running cell to complete the authorization process.

api_service_name = "youtube"
api_version = "v3"
client_secrets_file = "/content/drive/My Drive/creds_google.json"
scopes = ["https://www.googleapis.com/auth/youtube.readonly"]

flow = google_auth_oauthlib.flow.InstalledAppFlow.from_client_secrets_file(
    client_secrets_file, scopes)
credentials = flow.run_console()

The following code may prompt you to enable the YouTube API for the project in the Google Cloud Console the first time you run it. Follow the directions as needed.

youtube = googleapiclient.discovery.build(
        api_service_name, api_version, credentials=credentials)

Getting this far means we now have access to our YouTube data. Youtube Music basically stores all your likes in a playlist called “LM”. I’m presuming it stands for “Liked Music”. The API has a limit to how many results it would return at a time, so the following code will loop through the pages of the API response until we’ve collected the entire list.

def get_all_items_by_playlist(playlistId="LM"):
    request = youtube.playlistItems().list(
        playlistId=playlistId,
        part="contentDetails",
        maxResults=50
    )
    response = request.execute()
    all_items = response['items']
    while response.get('nextPageToken',"") != "":
        request = youtube.playlistItems().list(
          playlistId=playlistId,
          part="contentDetails",
          pageToken=response['nextPageToken'],
          maxResults=50
        )
        response = request.execute()
        all_items.extend(response['items'])
    return all_items
all_items = get_all_items_by_playlist()

As a quick check, I compared the length of this list with my expected value of 2255 to ensure I’m getting the correct playlist.

print(len(all_items))

Next I took a peek at the structure of the data by printing the first 5 items.

all_items[0:5]

It turns that “contentDetails” didn’t contain very many details about the content. There’s not enough information to identify a song based on this, so I reached back out to the YouTube API and asked for a “snippet” instead.

def get_music_metadata(playlistItem):
    request = youtube.videos().list(
        id=playlistItem['contentDetails']['videoId'],
        part="snippet"
    )
    response = request.execute()
    return response

test_md = get_music_metadata(all_items[1])
print(test_md)

At least now we have some information to work with here! There’s no clear cut way to pick out the song and artist though. Some YouTube videos have both the artist and the song in the title, but the majority only include the artist name in the channel. For my purposes, I’m going to treat the channel name up to the first dash as a proxy for the artist and video title as the song title. It’s not going to be 100% accurate, but hopefully will be close enough that Spotify can figure it out.

def guess_song_title(metadata):
  if len(metadata['items']):
    return metadata['items'][0]['snippet']['title']
  else:
    print('Failed to find song name for item:')
    print(metadata)
    return "Unknown Song"

def guess_artist_name(metadata):
  if len(metadata['items']):
    return metadata['items'][0]['snippet']['channelTitle'].split('-')[0].strip()
  else:
    print('Failed to find artist for item:')
    print(metadata)
    return "Unknown Artist"

print(guess_song_title(test_md))
print(guess_artist_name(test_md))

Having verified that I get a reasonable guess as to the song and artist associated with a sample video ID, I looped back through my list of all likes to acquire the rest of the dataset.

for i,item in enumerate(all_items):
    all_items[i]['metadata'] = get_music_metadata(item)
    all_items[i]['artist'] = guess_artist_name(all_items[i]['metadata'])
    all_items[i]['song'] = guess_song_title(all_items[i]['metadata'])

My results showed that my script failed on 1 item which seems to have been deleted, but that’s not a bad success rate considering how many it ran through. The next step is to see if we can find the songs on Spotify, so lets pull our credentials from the file we created earlier.

SPOTIFY_CREDS = "/content/drive/My Drive/creds_spotify.json"
with open(SPOTIFY_CREDS,"r") as f:
  spotify_credentials = json.load(f)

Now that we have our credentials in hand, we need to authenticate to the Spotify API like we did with Google earlier. Note that the redirect URI appears again here, so you may need to update this if you used something different.

market = [ "AD", "AR", "AT", "AU", "BE", "BG", "BO", "BR", "CA", "CH", "CL", "CO", "CR", "CY", 
      "CZ", "DE", "DK", "DO", "EC", "EE", "ES", "FI", "FR", "GB", "GR", "GT", "HK", "HN", "HU", 
      "ID", "IE", "IS", "IT", "JP", "LI", "LT", "LU", "LV", "MC", "MT", "MX", "MY", "NI", "NL", 
      "NO", "NZ", "PA", "PE", "PH", "PL", "PT", "PY", "SE", "SG", "SK", "SV", "TH", "TR", "TW", 
      "US", "UY", "VN" ]

sp_scope = "user-library-read user-library-modify"
sp_credentials = SpotifyClientCredentials(
        client_id=spotify_credentials["client_id"],
        client_secret=spotify_credentials["client_secret"])

auth_manager = SpotifyClientCredentials(client_id=spotify_credentials["client_id"],client_secret=spotify_credentials["client_secret"])     

sp_oauth = SpotifyOAuth(
    username=spotify_credentials["user_id"],
    scope=sp_scope,
    client_id=spotify_credentials["client_id"],
    client_secret=spotify_credentials["client_secret"],
    redirect_uri="https://localhost:8888/callback",
    open_browser=False
)

Like the Google API before it, Spotify will also ask us to provide an access token. We don’t actually have a web server set up to provide a nice user interface here, so you’re going to need to pull the access token from the address bar in your web browser.

When you run the following cell, it provide a link that opens in your web browser. When Spotify completes the authorization, it sends you to the redirect URI which will likely fail because your computer is probably not running a web server on the specified port (if you are, you might need to adapt a little here). This “Site can’t be reached” error is okay, but what we need here is the URL from the address bar. It should look something like this:

This whole URL will needed to be copy and pasted into this notebook to complete the authorization process.

sp_token = sp_oauth.get_access_token()

Now that we have an access token, let’s finish the the log in process and verify that we’re authenticated as the correct user.

sp = spotipy.Spotify(auth_manager=sp_oauth)
print(sp.current_user())

The next step is to use the artists and song titles we extracted from YouTube to find the corresponding track on Spotify. I’m going to put a dash between the artist name and the song title for my search string, then use the first hit in the results so see what I found.

def get_search_string(item):
  return item['artist']+" - "+item['song']

res = sp.search(get_search_string(all_items[0]), type="track", market=market, limit=1)
print(res)

The key piece of information we need here is the “track id”.

res['tracks']['items'][0]['id']

Since we need to do this for a lot of songs, we’ll wrap this process in a function and verify that it works.

def get_track_id(item):
  if item['song']=="Unknown Song" or item['artist']=="Unknown Artist":
    print(f"Skipping unknown item: {item}")
    return None
  search_str = get_search_string(item)
  res = sp.search(search_str, type="track", market=market, limit=1)
  if len(res['tracks']['items']):
    return res['tracks']['items'][0]['id']
  else:
    return None
my_track_id = get_track_id(all_items[0])
print(my_track_id)

Having extracted this track id, we can use it to add the item to our Spotify library. Before we do that, we should probably make sure it’s not already in the collection.

sp.current_user_saved_tracks_contains([my_track_id])

If it’s your first time running the above code, it should return “[False]” because we haven’t added anything to our library yet. Next, we’ll add it and try again.

def add_track_to_library(item):
  track_id = get_track_id(item)
  track_name = get_search_string(item)
  if track_id is None:
    print(f"Couldn't find track for: {track_name}")
    return
  if sp.current_user_saved_tracks_contains([track_id])[0]:
    print(f"Track is already liked: {track_name}")
    return
  print(f"Attempting to add track: {track_name}")
  sp.current_user_saved_tracks_add([track_id])

add_track_to_library(all_items[0])
sp.current_user_saved_tracks_contains([my_track_id])

The cell above should return “[True]” to indicate that the item has been successfully added to our library. The only thing left is to apply this process to the rest of our list. I’ve added a small sleep step to this loop to ensure that I stay within the Spotify usage limits

for i,item in enumerate(all_items):
  add_track_to_library(item)
  sleep(0.1)

Conclusion

That’s it! We’ve successfully migrated our YouTube Music Likes to Spotify Favorites. Now when Spotify tries to dynamically generate radio stations, it at least has some general information about what I’d like to listen to. There’s plenty of room for improvement in this code, but it got the job I needed done so I probably won’t be coming back to it any time soon.

If you’re looking for some project ideas to further develop your own understanding of these APIs, here are some potential modifications you might consider:

  • Try to improve the search criteria for better accuracy.
  • Try to create a new Spotify playlist from a Youtube Music playlist.
  • Try to sync Spotify favorites back to YouTube Music.
  • Try to implement this as a complete web application.
  • Try to analyze patterns in your music collection using data available through Spotify’s “Audio Features”.

There’s just so much you can do with these APIs! All it takes is a little imagination and some elbow grease!

Mathematics as a Creative Endeavor

I’ve recently started playing guitar again after years of watching it collect dust in the spare bedroom. It’s just not something that I found myself having time for while I was teaching. Now that the calluses on my fingers have started to return, I’m starting to remember what it was like to learn how to play guitar in the first place and noticing parallels with learning mathematics that I hadn’t considered before. I think I underestimated the degree to which mathematics is a performance art.

As a teacher, I think part of the difficulty with creativity is that it doesn’t fit well with our assessment driven school culture because it’s too abstract to measure. Our inability to grade creativity doesn’t mean we shouldn’t attempt to foster it in students. Instead, I’d argue that creativity is evidence that our very concept of grading is inherently flawed. We do a disservice to mathematics by presenting it as a mechanical process that is “right or wrong”. In reality, mathematics is about finding an artistic balance between “wrong” and “useful”. Mathematics is at it’s most beautiful when it precisely illustrates the boundary between what we think is true and the reality of truth.

When I first started playing guitar, I struggled to hold down the strings with sufficient force and they’d rattle against the frets as they vibrated. Obviously the solution to this was more practice, so my guitar teacher assigned my some warm-up routines. I was kind of reluctant at first, so he offered me some advice. He told me something to the effect of “when you see someone playing guitar, you don’t see all the thousands of hours they spent practicing to get there”. Now, more than 25 years later, these riffs are still one of the first things I find myself doing when I pick up a guitar. Their memory is ingrained in my fingers.

There’s a big gap between this rote memorization of exercises and producing an original work. I don’t think I would have ever developed the capacity to improvise on the guitar without practicing a variety of different scales, but also know that no amount of repetition would magically turn me into a composer. To get better at creating music, one must practice creating music. If I’m being perfectly honest, most of that music I create isn’t even that good, but I’ve learned to accept that’s just another part of the creative process. Nobody hears all the flops that had to be written before making that one hit record. Time is a necessary element for any creative process.

As a teacher, the long term effects of my mathematical practice were readily apparent. I could take one glance at a high-school level problem and mentally calculate the solution so quickly that it seemed like magic. Sometimes math needs that bit of awe that you get when you hear a shredding solo from Steve Vai. At the same time, you don’t want to intimidate a new learner either. No aspiring guitarist starts out with “Teeth of the Hydra”, but there’s something special about seeing what’s possible. Every artist has some source of inspiration that drives them.

More practically, the real application of this procedural fluency was that it allowed me to focus my attention on the student’s model of a math problem rather than my own. Much like playing in a band, communicating mathematical ideas to others necessitates the ability to listen closely and willingness to adapt. Not only do the band members communicate with each other to hold tempo and maintain harmony, but the band as a whole feeds off the audience’s energy as well. Having a clearly identified target audience gives direction to the creative process, even when that audience is only oneself. Creativity depends on a foundation of shared cultural experiences between the artist and audience.

Cultural conventions in music provide another essential component of creativity: constraints. It’s not required that a song to be 3-5 minutes long, but knowing that that’s the average length for radio play allows a band to focus the creative effort on what to fill those airwaves with. Forced limitations of scope can actually encourage creative solutions by reducing the burden of what needs to be created. Most academic papers follow a standardized format because it allows the writer and reader to focus on the content instead. Classroom routines allow teachers and students to focus on addressing specific learning objectives. This must be done in careful moderation though, because too many constraints can stifle creativity by denying agency.

I think this sense of agency is perhaps one of the key elements that made my experiences with math differ from the norm. I enjoyed doing math so I constantly sought out new opportunities to experience it. I didn’t come into a math class asking “when am I ever going to use this?” because I already had use cases in mind. I think that one of the great travesties of education is that the system doesn’t allow for students to assert full ownership over their own learning. The idea of forcing every student to learn guitar whether they want to or would be ludicrous, but our society confers special treatment to mathematics as a result of it’s perceived pervasiveness. Creativity cannot manifest without providing the creator with choices to make.

I’ve come to terms with the fact that I can’t produce creative work at will. Whether it’s math or music, I don’t know when that flash of sudden insight will occur or if it even happen at all. It’s out of my control. What I can do is to assert control over my environment to set up favorable conditions for that spark to ignite:

  • I can set aside time to focus on the creative process.
  • I can provide myself with a safe space to make mistakes.
  • I can surround myself with prior art for inspiration.
  • I can learn more about the social context I share with my target audience.
  • I can set additional constraints on the outcome to make the task more manageable.
  • I can decide how much of myself I’m willing to put into the work.

In a society that’s increasingly driven by automation, I think there’s going to be a growing need for fostering creative thinking in schools. This change will be difficult and will require us to change how we conceptualize learning. Perhaps a place to start is to rethink math classes to be more like a course in music appreciation. Much like it would be impossible to understand the impact of punk rock or hip hop without understanding the social-political climate it developed in, we can’t expect students to understand the impact of mathematical developments if we take them out of context. We need to allow students space to enjoy math in its natural environment instead of distilling it down to a sequence of mindless practice exercises. Raising a generation of creative problem solvers will require an educational system that places the needs of students over the ideals of society.

Research Statement

I’ve always been a self-driven learner, but I’m starting to understand that there are limits to what I can learn on my own. As a teacher, I found myself working hard to establish a “community of learners” in my classroom. Now I’m finding myself at a crossroads where I could really benefit from a learning community of my own. Perhaps by laying out my ideas of interest here some potential mentor or peer group might find me.

My primary area of interest is in the application of technology to improve educational outcomes. I’ve taken a multi-disciplinary approach to building this expertise in both breadth and depth, building on my dual undergraduate degrees in Mathematics and Psychology. My graduate work in Learning and Technology focused on the use of online discussion boards in mathematics and helped prepare me for the emergency learning transition of early 2020. Since then, I’ve developed an interest in the applications of machine learning to help induce positive structural changes in education.

Artificial Intelligence (AI) is rapidly advancing and the field of education stands to be dramatically changed as a result. The question is not if AI will be used in school, but rather when and how. The existence of applications like PhotoMath have already shifted the focus dialog in my math classes from “the result” to “the process”. Large Language Models such as ChatGPT will revolutionize the way we think about student writing the same way. Rather than such tools being banned from classes, schools should instead focus their efforts on making sure students are equipped to collaborate responsibly with AI models.

The field of education presents some unique challenges for AI. The haphazard implementation of AI could cause immense harm, so the first order of business is to develop systems of safeguards against misuse. While AI grows in power with massive data sets, schools have competing legal obligations to be met regarding data privacy, transparency and ethics. Schools will need to have traditional engineering methods in place to detect algorithmic biases with regard to legally protected statuses such as race or gender before they can reliably depend upon consumer AI solutions. Equity needs to be built into AI from the ground up.

One of the challenges posed by AI produced content is our inability to distinguish it from human produced content. Our best tool for detecting AI produced content is to train another AI to do the job for us. These adversarial agents complement each other when trained in parallel. As the AI classifier that detects whether or not an artifact is human produced gets better, it forces the content producing agent to behave more human-like. These two AIs are designed to compete with each other, but the feedback loop created when they interact together allows them both to learn more efficiently.

Applying AI in education is going to require a whole ecosystem of agents who compete or cooperate to provide checks and balances on the values we which to instill. The first step towards this is to looking at education through a lens of game theory that critically examines the reward structure of the school. The challenge is creating a set of macro-level rules for interactions between AI and human agents that promotes collaborative behavior across the system. We need to shift the discussion from “how do we prevent students using AI to cheat?” to “how do we align our assessment methods with actual student growth?”.

One of our highest priorities with AI should be the development of an AI learner advocate. Ideally, the system would allow students and parents to articulate their long-term personal learning goals through natural language in an Individualized Education Program (IEP) and the AI would automatically configure itself as needed to monitor and advance those objectives. For example, an algorithm might be programmed to automatically alert teachers if they accidentally forget to include alt text in an assignment with visually impaired students in the class. It’s also important that students maintain a voice in the learning process, and AI may provide a powerful tool for students who have yet to learn how to self-advocate.

In the long term, AI could also provide considerable time savings to teachers by producing “just-in-time” lesson plans based on the available data. Writing lesson plans is a task that is often more tedious than it is difficult — assuming it’s not a direct copy of one from last semester. We’re reaching a point where AI can potentially automate the process of aligning experiences with instructional objectives, measuring progress relative to the course calendar, recommending accommodations based on IEPs, and formatting this data in a standardized template for administrators. This would free up more time for the teacher to spend customizing the learning experiences for their specific classes.

From a mathematical perspective, algorithmic lesson generation bears some key structural similarities to the “Many-Armed Bandit Problem” in probability theory. Consider the teacher like a gambler in a casino and the slot machines as various potential lessons that can be assigned to a student. The teacher wants to select the experience with the highest odds of paying-out for that student, but not every student will respond to that particular experience in same way. For a teacher to maximize the learning potential for all students, they must strike a balance between exploiting lessons that tended to work in the past with exploring new methods they have yet to try. The more information a teacher has about the student, the more reliably they’re able to predict what might work.

The fact that confidential health information about a student is often directly linked to academic accommodations presents a unique privacy problem for AI. In much the same way that Security needs to be applied across Development and Operations, so too must Accessibility be asserted across educational organizations. A substitute teacher doesn’t need to know every student’s complete medical history, but a classroom AI might be able provide them with the information that “students need a break” without revealing the fact that Johnny has ADD. There is a delicate balance to be made between controlling access to sensitive data on a need-to-know basis and using that data to provide teachers with actionable insights.

As a former teacher, the idea of a computer listening and watching everything I do in the classroom is terrifying. We cannot permit AI to become an electronic police force is schools. At the same time, I also know that the amount of data produced in a live class is more than any single person could analyze on their own. The subtle timing of a student’s confused facial expression can often speak volumes about that student’s understanding. I would not the least be surprised if an AI could even out perform me on this task, but also know there will always be an unavoidable risk that this AI might be wrong and cause irreparable harm. The tech industry mantra of “move fast and break things” is an inappropriate philosophy for schools.

Instead, education must adopt AI slowly and with confidence. Educational data must be treated with the same level of careful stewardship as healthcare data. Researchers need to gradually integrate AI through a series clinical trials and monitor the system as a whole for changes. Much like a teacher, AI needs to build a relationship of trust with all of the stakeholders in the environment. Students and parents need to know the AI is working for them and not against them. Teachers and administrators need to have the power to overrule AI assessments where appropriate.

I realize that these scenarios might seem like widely disconnected fields, but I’m finding that they have some common mathematical threads. I accepted a long time ago that I might spend a lifetime working on Millennial Problems I’ll probably never solve and instead try to focus on making progress in small steps. I’m working to further my knowledge of Category Theory because I’m finding the language of maps and objects to be extremely valuable in modeling complex systems. I feel there’s a deep connection between Complex Analysis, Linear Algebra, and Topology that I’m just shy of understanding. I’m also starting to realize a need for better Statistical tools, particularly in the properties of different Probability Distributions used for modeling risks.

I’m very much open to graduate level research opportunities in Mathematics or Computer Science that would advance my understanding of the topics described above. If you’re in a department working towards these high level goals and have an opening for a research assistant, please feel free to reach out to me through any of my social media profiles.