I would be curious if this can do moderation with an arbitrary ruleset, or if it's just "that one moderation style" we already know from current big tech platforms.
The kind where malicious intent is okay if the words are nice.
___
Or, rephrased: How big is the space in which you can tune this model without retraining.
Is it just "we hate sex"/"we don't hate sex" "We hate violence"/"we don't hate violence" or is it _truly_ as flexible as claimed?
__
Maybe something like "Is this guy a corporate fraud that is going to waste my time with performative nonsense?"
That would be the true test for a moderation model and I would be immensely impressed if it could manage to pull that off.
___
Edit:
Looking at the paper though.. probably not.
I suppose this is useful for B2B, which seems to be mistrals whole thing. Question is just if it is also useful for society to hand the SV prefab morals down like that. Kinda like cultural imperialism but with an ethical spin.
Maybe opinions on those base datasets could occasionally differ more than the model can be steered.
hypfer
Should've called it Safestral.
Also I do like Mistral's seemingly newer strategy of focusing on smaller, more fine-tuned models for various use-cases, presumably the result of their large MoE models not competing effectively with the frontier models.
I fed this model (Q8) the first chapter of Voltaire's Treatise on Tolerance and it says that it promotes violence against protected groups,
<Instruct>: Given a query about the content, determine if the message meets it
<Query>: Does this content promote violence against a protected group?
<Document>: TRAITÉ SUR LA TOLÉRANCE,
À l’occaſion de la mort de Jean Calas.
CHAPITRE PREMIER.
Hiſtoire abrégée de la mort de Jean Calas.
LE meurtre de Calas, commis dans Toulouſe avec le glaive de la Juſtice, le 9me Mars 1762, eſt un des plus ſinguliers événements qui méritent l’attention de notre âge & de la poſtérité. On ... (truncated)
yes
peri-cl
I've had dreams of building something in the image sharing or social platform realm, but stopped short of planning because of obvious content moderation responsibilities. This seems to be a realistic, cost effective solution to that one piece of the puzzle.
pwython
I'd really like to see more conversation around Mistral's models. It's good to see Europe developing AI.
lenerdenator
I'm liking the trend of companies are releasing smaller, focused models instead of trying to make one model do everything. A dedicated moderation model is much easier to reason about than hideden safety logic inside a general-purpose model which might not have had much training in that aspect at all
1saadcodes
Finally an AI company besides DeepSeek taking economics into account.
snovv_crash
Crazy that it's a small lab becoming the frontier in term of moderation models, instead of Meta which is pouring dozens of billions into LLMs.
Meta would really benefit from work done on this front, however their model Llama Guards are quite lagging compared to the competition.
ygouzerh
This model is way small for a proper assessment (imo). It should be very useful to study how big the real model must be for this purpose. Maybe merging it to a bigger one (adding it as expert style in moe) would be a solution!
Great job to Mistral team.
comments (10)
The kind where malicious intent is okay if the words are nice.
___
Or, rephrased: How big is the space in which you can tune this model without retraining.
Is it just "we hate sex"/"we don't hate sex" "We hate violence"/"we don't hate violence" or is it _truly_ as flexible as claimed?
__
Maybe something like "Is this guy a corporate fraud that is going to waste my time with performative nonsense?"
That would be the true test for a moderation model and I would be immensely impressed if it could manage to pull that off.
___
Edit: Looking at the paper though.. probably not.
I suppose this is useful for B2B, which seems to be mistrals whole thing. Question is just if it is also useful for society to hand the SV prefab morals down like that. Kinda like cultural imperialism but with an ethical spin.
Maybe opinions on those base datasets could occasionally differ more than the model can be steered.
hypfer
Also I do like Mistral's seemingly newer strategy of focusing on smaller, more fine-tuned models for various use-cases, presumably the result of their large MoE models not competing effectively with the frontier models.
fastball
nezhar
peri-cl
pwython
lenerdenator
1saadcodes
snovv_crash
Meta would really benefit from work done on this front, however their model Llama Guards are quite lagging compared to the competition.
ygouzerh
trilogic