The algorithm is barely IP, and the data is the bigger part of it's success.

ChatGPT is a reinforcement learning tuned transformer. The ideas and architecture it's built on aren't proprietary. The specific parameters are, but that's not actually that important. The size and number of layers, for example. Most people in ai can make some assumptions. Probably ReLU, probably Adam, etc. Then there are different knobs you can twiddle and with some trial and error you dial it in.

The size and quality of your training data is way more important, and in the case of chatgpt, so is your compute power. Lots of people can design a system that big, it's as easy as it is to come up with big numbers, but training it takes a ton of compute power, which costs money, which is why just anyone hasn't already done it if it's so easy.

It should also be said that GPT is a bit of a surprise success. Before models this size, it was a big risk. You're gonna spend millions to train a model, and you won't know until it's done how good it will be.

Most advancements in AI are open source and public. Those all help advance the field, but at the same time, it's also about taking a bit of a risk, and waiting to see how it pans out before taking the next risk.

Also, there's transfer learning. If you spend a hundred million training a model, I can use your trained model and a fraction of the money to make my own .

It's like if you laboriously took painstaking measurements to figure out an exact kilogram and craft a 1kg weight. You didn't invent the kilogram, difficult as it was to make it. If I use yours to make my own, I'm not infringing on your IP.

I'm a little in between, here. I've posted OC a couple of times, and I'm working on a new one currently.

I for one do think aesthetics should be part of it, absolutely. Having posted OC before, though,you will get a lot of pushback for aesthetic choices - if your colors don't perfectly interpolate or anything can be interpreted as not strictly accurate. There's a degree to which accuracy and aesthetics are at odds. For example, a pie chart can be perfectly accurate, but if certain colors are too similar or one color stands out without good reason, you will be accused of manipulating data or not presenting it honestly.

On the other hand, I also think some data just are beautiful, regardless of aesthetics. Interesting data are fun to look at on their own, and the data itself is part of the beauty, like "r/oddlysatisfying" I think sometimes a post isn't about the data visualization being beautiful , but the data itself.

Ultimately, though, it's a matter of voting, particularly on new/rising posts. Given that only about one in a thousand viewers or less of a post actually up or downvote, individuals have a lot of power to reshape the sub, by voting in new/rising. If more aesthetically-minded posts are lifted, the OC will follow suit.

Machine learning, as people have pointed out, is broad. However, I think that understanding gradient descent in general really gets to the heart of most new applications (especially neural networks).

Gradient Descent is kind of like a game of hotter/colder. You start by walking in a completely random direction, and then someone tells you you're either getting warmer or getting colder.
A neural network starts similarly, taking it's input and doing a bunch of random multiplications and getting random output. Then you tell it what the answers should have been and it knows how far off it was. Then it goes back to all those random variables (parameters) and calculates how much each one contributed to it being wrong, and adjusts them ever so slightly so that they would have produced a better result.

Congrats to your cousin!

Weird choice, Doug.

First I've heard of that sub - feel free to cross post.

Time is a useful dimension for data visualization.

>1.5% is significant from a possible sample of 000s of names

A percentage is already "out of". It's already a ratio.

How would fearing a loss of identity lead to abandoning that identity and inventing a new one?

Fearing a loss of identity is not the same as trying to establish an identity. It's very much the opposite.

I see no compelling reason for the hypothesis you suggest.

Just because it's less unique doesn't mean it's not done for uniqueness. "Paisley" is already a word, "Paizleigh" is a strategy to be unique. It's certainly not traditional or closed minded.

It's also still really unique. At its peak, "eigh" appears in 1.5% of new baby names in Mississippi.

Finally, it obviously doesn't matter if income is less predictive of "eigh" than race, education, or place of residence. The question is whether it's more predictive than political affiliation.

If you had explicitly said something about income, then my statement "I imagine that's who you're picturing" would have instead been "I understand that's who you're picturing".

By what means would you hypothesize a causal link from education level to use of "eigh" in a name?

That's good to know. It does say it's a 100% sample, but that would explain why the lowest count I can find is 5. Do you have a better source?

Whats the Canadian equivalent of a social securiteigh?

I'm working on another project that I hope will shine some light on that.

Are more popular than what? Those names don't contain "eigh". What do you think we're talking about here?

More popular than their variants that do contain "eigh"? This is showing the popularity of names including "eigh" relative to all baby names by year. "All baby names" includes phonetically similar names that don't include "eigh".

Again, I wouldn't see any reason to correlate it with education level or rural people. The map really only shows that it's a southern thing.

Sure the south could have more rural white without a college education, but it also has more fresh peaches and warmer weather. You can't connect all of those things.

Oh that's weird. I'll go back and check my data. Obviously one of those is wrong.

Yeah, I was wrong on leighanne - not sure what happened. Ran it again and it's Leighton in NY 1914.

I disagree... On one hand, it is associated with the south, but on the other hand, I wouldn't expect name uniqueness to correlate with "reverence for tradition".

Conservatism is associated with a distaste for change and abhors anything "newfangled". I don't see how that goes hand in hand with inventing new names.

Being poor or lower class, on the other hand, likely correlates with a desire for uniqueness or any form of status, so I would entertain that hypothesis, and I imagine that's who you're picturing.

But the percentage of Republicans hasn't changed and so wouldn't explain the growth in popularity, which suggests any correlation would be indirect. I expect the growth in popularity is due to the internet and contrary to rural people. In my parents life, there were historical figures, some people on TV, and the people who lived in your town. No one else really existed. We meet and interact in social circles many orders of magnitude larger, which I think both increases the amount that we value uniqueness, and raises our threshold for what qualifies. The bigger your "world", the more unique a baby name you'll likely choose.

EDIT: not complaining, but the downvotes surprise me. I'm not sure if they're from members of the political left who don't like "eigh" or from members of the political right who feel I've taken a reductive stance on their beliefs. I just don't see a compelling hypothesis for linking republicans and "eigh" and I do find it compelling to believe more conservative people would tend to more conservative names. I don't see that the data are affected by the southern strategy, or anything other than southern states.

Yeah, I added the part at the bottom at the end, but scrapped the y-axis because it was too crowded and jumped around too much, and I figured people cared more about the relative change than the actual value.

Data from data.gov 100% sample of social security application names 1910-2020*.

Made with Python (w/ plotly, PIL)

*There were no names with 'eigh' in 1910

EDIT: Forget Leighanne. Leighanne sucks. The first use of EIGH not at the end was way back in 1914, 6 boys names LEIGHTON born in, you guessed it, NY.

Interesting - seems it's mostly just trying to fit one experts idea of what valence means, so we'd have to ask that dude what features contribute to valence.

But... aside from the fact that it's relatively old and predates a lot of the better language models we have today, I think we can also conclude it's likely not looking at lyrical content by the inclusion of both Hey ya and Pumped Up Kicks at the top of the valence scores.

Hmm... I tried to find out more about the "valence" attribute from spotify. I assume both of these values are coming from machine learning algos. Spotify describes valence as how "positive" something feels, but the word valence comes from chemistry - the linking between atoms. It's also used in linguistics to describe the linking of terms (in "charlie gave the big, jolly, man apples", 'gave' is linked to 'charlie' and 'apples' and 'man', 'the', 'big' and 'jolly' are also linked to 'man', but not to 'gave' as they refer to 'man' not the act of giving, so 'gave' is connected to 3 words, valence of 3, man is connected to 4 words, valence of 4).

Since the linguistic definition of valence doesn't at all seem to fit 'positivity', I assume valence here is tonal or rhythmic, but it's not a common descriptor in music generally. It could be rhythmic synchronization, or an inverse of syncopation, but that wouldn't generally make something feel positive, so my best guess is tonal, and it's a machine learning algorithm trying to fit the pitches to the tonic / major scale. Any song in a major key can be rewritten exactly the same in a minor key. A minor and C major use all of the exact same notes, but if the song tends toward C as "home" it's Major and happy sounding, if it tends toward A as "home" it's minor and more sad or whatever. You can't just have an algo look at what notes are used to determine if it's major or minor, you would more likely look at the distribution of how the notes are used, which is something I could see describing as valence. Plus, Major and Minor are binary, but there's really a continuum. Mary had a little Lamb is very very major. Moonlight sonata is very very minor. So I think here we're seeing songs growing increasingly less major. Specifically less major, though - there are other modalities of music outside of Major and Minor, and a single metric wouldn't really capture all of that - less major doesn't necessarily mean more minor, although we could generally assume it mostly goes that way.

That's a completely different statement.

In your first statement, you compared experts to omniscience. Your argument can be interpreted as "experts are not always correct, therefore we shouldn't value their opinion"

The rebuttal was that instead of comparing experts to omniscience, the more appropriate comparison is to the alternative: non-experts. Neither is always correct, but those are the options, and the experts are preferable.

You then mischaracterize this, "experts are correct more often than non-experts" as "experts are correct more often than they are incorrect". That's an entirely different statement. It is not the statement being made in the comment you're replying to.

Was that on purpose, or a mistake?

I don't see how you would train them that way - you can't use the output of a discriminator as the input of a generator - that wouldn't get you what you want. You could train them in parallel, one network and discriminator doing only b&w restoration, and the other doing only colorization.

The way images work and the eye (part of the science behind why jpeg is so useful) is that we're much more sensitive to luminance information than color information. You could take the output of colorized image in hsl color space and replace the luminance with that of the generated restored photo. Doing it this way, you could force the separation of two generators using only one discriminator, as well - one generator only affecting the hue and saturation of the final image, and the other only affecting the luminance.

That said, with the more recent breakthroughs, it seems that networks are proving more successful as generalists than specialists. For example, it's believed that whisper performs better on each language because it's trained on all languages, as counter-intuitive as it may seem.

All you need to know over time is the pitch being played, which is a frequency. The audio file represents a waveform, and all you need to know is the frequency of that waveform over time. There's no need for anything sequential. 440Hz is "A" no matter where it comes in a sequence. It's A if it comes after C, and it's A if it comes after F#.

A sequential model might be useful for natural language, for example, because meaning is carried between words. "Very tall" and "Not Tall" are different things. "He was quite tall" and "No one ever accused him of not being tall" are remarkably similar things. Transcribing music is just charting the frequency over time.

That said, you cannot get the frequency from a single data point, so there is a somewhat sequential nature to things, but it's really just that you need to transform the positions of the waveform over time into frequency, which the fourier transform does. When music visualizations show you an EQ (equalizer) chart to go with your music, this is what they're doing - showing you how much of various frequencies are present at a given time in the music, using a FFT. A digital equalizer similarly transforms audio into a frequency spectrum, allows you to adjust the frequency spectrum, and then transforms back into a waveform.

I'm a little unclear - there are three different things you might be trying to do here. The first would be transcription - taking an audio file and interpreting it into notes. That wouldn't typically require deep learning on it's own, just a fourier transform. The second would be isolating a specific instrument in an ensemble - finding just the recorder in a collection of different instruments all playing different things. The third would be generation, inferring unplayed future notes based on previous notes.

Are you wanting to transcribe, isolate, generate, or some combination?

I'm thinking you're wanting to transcribe. If that's the case, FFT (fast fourier transform) would be the algo to choose. If you google "FFT music transcription" you'll get a lot of info. https://ryan-mah.com/files/posts.amt_part_2.main.pdf