S1: I'm not completely sure I'm right about this, but I've
S1: never been a big believer in training custom models. I've
S1: also never believed in fine tuning going all the way
S1: back to 2023. My intuition has always pushed me towards
S1: the best state of the art model possible, combined with
S1: context management. I just finally crystallized my reasoning around this.
S1: Anytime you think you're using a small model for a
S1: small task, there's usually a whole lot more going into
S1: a given decision than just that individual area of expertise.
S1: For example, labeling emails, writing reports, processing security events, searching
S1: for threats on a network. On one hand, I think
S1: these are specialized, but the fact is, the smarter and
S1: more experienced a human is who has this expertise, the
S1: better job they're going to do. This is because most
S1: specialized tasks still benefit from the general life experience of
S1: the person doing the execution. This is why I think
S1: the future is not a whole bunch of extremely small,
S1: specialized models throughout the enterprise. I think what's far more
S1: likely is more of an opus, sonnet haiku model, where
S1: the best of the best just keeps coming down in price,
S1: including going into open source. And those smaller models are
S1: used in conjunction with context to perform all the different
S1: tasks in an organization at much lower cost. But I
S1: think they'll still be extremely general models, not tiny and
S1: narrow custom ones. I think the Tldr here is when
S1: you think you're doing a narrow task, that narrow task
S1: is actually benefiting from a ton of general experience. And
S1: I think this applies to humans, and I think it
S1: also applies to models. I'm not completely convinced of this.
S1: I'm about 70% sure. But yeah, I think this is
S1: the way it's going to go.