TinyButStrong and mbstring.func_overload

Hi,

I'm trying to get TBS working under utf-8.  We want to turn on func_overload so that we don't have to go through the source of our websites to manually convert everything over.

When I set func_overload to include the str* functions, the TinyButStrong tags are not processed and just appear as text within the HTML - for instance [block.savedDetails]. 
With  func_overload set to 4 (ereg functions only) the page renders fine.

Here's our mbstring settings from php.ini:

mbstring.language        = Neutral; Set default language to Neutral(UTF-8) (default)
mbstring.internal_encoding    = UTF-8  ; Set internal encoding to UTF-8
mbstring.http_input = auto
mbstring.http_output          = UTF-8   ; Set HTTP output encoding to UTF-8
mbstring.encoding_translation = On
mbstring.detect_order = auto
mbstring.substitute_character = none;
mbstring.func_overload = 6
mbstring.script_encoding=UTF-8

All of our templates' file encodings are in utf-8
Our database is utf-8 as are the database connections
We're using PHP 5.3.10, Mysql 5.5, Apache2.2, Debian Squeeze, TinyButStrong 3.7.0

I appreciate I've not posted any code, but that's because it's hard to pin down at which point in the codebase this is happening.  Rather, does anyone have any ideas or suggestions as to what could be causing this please?

Thanks in advance

Dan

Hi Dan,

This is quite strange. I'll make some test in order to reproduce the behavior and understand what's happen.

hi Dan,

I didn't succeed in reproducing the problem.
I tried with PHP 5.3.5, TBS 3.7.0, with a simple utf-8 PHP script merging an utf-8 template.
There was no proble with the values "mbstring.func_overload = 2" and "mbstring.func_overload = 4".

Can you check that the error cannot come from another part of the script. Did you turn the "error_reporting" option the E_ALL ?

Hi Skrol,

I'll create a simple test page with a couple of templates to try to reproduce the problem in a simple way and post back here.

Thanks

Dan

Ok, or send it to me by email.

Hello,

I’m using the last version of TBS (3.9.0) and OpenTBS (1.9.0).
This problem with mbstring.func_overload has not yet been resolved. I recently activated mbstring.func_overload. TBS started to give me errors like:
TinyButStrong Error OpenTBS Plugin: Signature of Central Directory Header #1 (file information) expected but not found at position 54519 (h:d4f7). The process is ending, unless you set NoErr property to true.

Would you correct this incompatibility of TBS with mbstring.func_overload?

The problem disappear if i set mbstring.func_overload to 4 instead of 6.
So I suppose it is the Overload str*() functions (value 2) that are the problem.
But maybe it is not only the Overload str*() functions but a combination of them with other mbstring parameters that cause the problem)

My settings for all the mbstring parameters are now (but I would like to be able to put again the func_overload to 6)

[mbstring]
; language for internal character representation. Default is "Neutral" (UTF-8)
; http://php.net/mbstring.language
;mbstring.language = Japanese
;mbstring.language =

; internal/script encoding.
; Some encoding cannot work as internal encoding.
; (e.g. SJIS, BIG5, ISO-2022-*)
; http://php.net/mbstring.internal-encoding
;mbstring.internal_encoding = EUC-JP
mbstring.internal_encoding = UTF-8; JM

; http input encoding.
; http://php.net/mbstring.http-input
;mbstring.http_input = auto
mbstring.http_input = auto; JM

; http output encoding. mb_output_handler must be
; registered as output buffer to function
; http://php.net/mbstring.http-output
;mbstring.http_output = SJIS
mbstring.http_output = UTF-8; JM

; enable automatic encoding translation according to
; mbstring.internal_encoding setting. Input chars are
; converted to internal encoding by setting this to On.
; Note: Do _not_ use automatic encoding translation for
;       portable libs/applications.
; http://php.net/mbstring.encoding-translation
;mbstring.encoding_translation = Off
mbstring.encoding_translation = On; JM

; automatic encoding detection order.
; auto means
; http://php.net/mbstring.detect-order
;mbstring.detect_order = auto
mbstring.detect_order = auto; JM

; substitute_character used when character cannot be converted
; one from another
; http://php.net/mbstring.substitute-character
;mbstring.substitute_character = none;

; overload(replace) single byte functions by mbstring functions.
; mail(), ereg(), etc are overloaded by mb_send_mail(), mb_ereg(),
; etc. Possible values are 0,1,2,4 or combination of them.
; For example, 7 for overload everything.
; 0: No overload
; 1: Overload mail() function
; 2: Overload str*() functions
; 4: Overload ereg*() functions
; http://php.net/mbstring.func-overload
;mbstring.func_overload = 0
mbstring.func_overload = 4; JM

; enable strict encoding detection. permet de détecter des caractères UTF-8 malformés. à essayer s’il y a des problèmes d’interprétation de charset
;mbstring.strict_detection = Off

; This directive specifies the regex pattern of content types for which mb_output_handler()
; is activated.
; Default: mbstring.http_output_conv_mimetype=^(text/|application/xhtml\+xml)
;mbstring.http_output_conv_mimetype=


Thanks for your attention.

Hi,

Thank you for this well detailed description.

Using "mbstring.func_overload" with the flag 2 (values can be 2, 6, 10, ...) bring some problems with PHP code dealing with binary contents.
Functions strlen() does not return the binary length anymore, but the UTF8 length. And substr() cannot find a binary sub-string neither.
But OpenTBS actually need to read binary content of the zip archive. (It use the TbzZip class for this purpose).

The compatible code for strlen() that supports "mbstring.func_overload" becomes :
which is quite heavy to carry in an application.

I cannot see for now a smart solution for dealing binary content in PHP while "mbstring.func_overload" is activated with flag 2.

PHP doc :
http://www.php.net/manual/en/mbstring.overload.php

I've faced the same problem and found a solution:

Tanks !