Ok, I looked:
1. I have it in a separate .dll, not in the hosting (web) project.
2. In visual studio, where the behavior extension is used (in the example below this is the "<silverlightFaults/>" element) it gets underlined with a blue squiggly line and the tooltip says it can't be found, however at runtime it works like a charm.

Here's my web.config relevant part from the hosting application:

<system.serviceModel>
<behaviors>
<serviceBehaviors>
<behavior name="SDS.WCF.ServiceImplementation.DataService_Behavior">
<serviceDebug includeExceptionDetailInFaults="true"/>
<serviceMetadata httpGetEnabled="true"/>
</behavior>
</serviceBehaviors>
<endpointBehaviors>
<behavior name="SilverlightFaultBehavior">
<silverlightFaults/>
</behavior>
</endpointBehaviors>
</behaviors>
<services>
<service behaviorConfiguration="SDS.WCF.ServiceImplementation.DataService_Behavior" name="SDS.WCF.ServiceImplementation.DataService">
<endpoint behaviorConfiguration="SilverlightFaultBehavior" address="" binding="basicHttpBinding" name="DefaultEndpoint" bindingNamespace="urn:Ssmt.SilverlightDataService.ServiceContracts:v1" contract="SDS.WCF.ServiceContracts.IDataServiceContract"/>
</service>
</services>
<extensions>
<behaviorExtensions>
<add name="silverlightFaults" type="SDS.WCF.ServiceImplementation.WCFHelpers.SilverlightFaultBehavior, SDS.WCF.ServiceImplementation.ServiceImplementation, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null"/>
</behaviorExtensions>
</extensions>
</system.serviceModel>