Topic: Syntax Highligting for Code

[G_G]

I found this SMF mod. http://custom.simplemachines.org/mods/index.php?mod=2109
It will format the code to match syntax colors and stuff. It would be used like this [code=alias][/code].

It includes all the supported languages, include Ruby. I think it'd be interesting to have. And as far as I know, the normal [code][/code] still works as well.

[Blizzard]

I've already tried to add that syntax highlighting a few times, but every time it was a complete disaster to implement or it would break everything else. I was going to try again the next time I have time.

[Holyrapid]

Well, if you get it to work, I'd say that it would make finding mistakes in codes easier, at least a little...

[Blizzard]

You don't type code in the forum. And the syntax highlighting would only be applied to already posted stuff, not to the text you are typing.

[G_G]

Theres this project called GeSHi. There used to be an SMF mod but due to GeSHi's license it was removed from the SMF site. However, I have found a copy of it on the internet, if you want it I'll PM it to you.

More on GeSHi http://qbnz.com/highlighter/

[Blizzard]

I've tried to integrate it today and I found two problems. The first one was that it simply didn't work. :/ The second one is this: http://code.google.com/p/syntaxhighlighter/wiki/Overview *points to part with performance*

EDIT: I added the Select option for code, though.

[G_G]

Woo! Bans over! Anyways, theres still this.

Theres this project called GeSHi. There used to be an SMF mod but due to GeSHi's license it was removed from the SMF site. However, I have found a copy of it on the internet, if you want it I'll PM it to you.

More on GeSHi http://qbnz.com/highlighter/

EDIT:

Code: [Select]

Sexy






Select







Button





*orgrasms*

[Blizzard]

Yeah, I know. I'll try that one another time.

[G_G]

That GeSHi syntax works pretty good. Pretty fast too.
http://www.decisive-media.net/index.php?topic=106.msg646#msg646

[Blizzard]

I wasn't able to integrate this syntax highlighting mod due to some problems when I tried to a few months back. But Ryex was able to backport a mod from SMF 2.x which I will add over the course of the next few days. You can all thank Ryex by giving him a blowjob. Or even better: give me a blowjob. I love you all. <3

[Ryex]

The things I do when I get board but don't really want to work on the things I need to...

[G_G]

Glad to finally see this happening. Pretty awesome. :3

[Blizzard]

I added it. I had to first fix a few small problems (and a big one) with the add-on and afterwards adjust the style sheet, but after that everything is the way I want it.

[winkio]

Just checked it out, looks nice.  Good work

[Blizzard]

I just finished applying everything on the dark themes, too. Now I only have to figure out why the "Select" button doesn't appear anymore and how to add back where it says "Code" is no name for the tag was specified.

[Ryex]

looking pretty good if I do say so myself

[ThallionDarkshine]

It looks like the <a> tag is there, its just that it has no text in it. Maybe something got messed up in the php or javascript and its not returning the text for the select button.

[Blizzard]

Does the code box appear in the center to anybody else except me? If yes, which browser are you using (I'm on Opera)?

[winkio]

The box itself is stretched across the entire width of the post, and the text is aligned to the left, on Chrome.

[ThallionDarkshine]

It's the same in firefox as in chrome.

[Ryex]

Welp, it looks good on the dark theme now. Enjoy folks. Blizzard probably did just as much work as I have on this.


for your information it attempts to auto detect the language but it can get confused if the section is small you can help it pick the language with the code tag property

[code=ruby][/code] Ruby
[code=php][/code] PHP
[code=cpp][/code] C++
[code=cs][/code]   C#

ect.

EDIT:
At least it would if the code tag still put that value after the :  is the code tag header...
FIX IT BLIZZ

(click to show/hide)

Code: (php) [Select]

$codes[] = array(
'tag' => 'code',
'type' => 'unparsed_content',
'validate' => isset($disabled['code']) ? null : create_function('&$tag, &$data, $disabled', '
global $txt;
// select all and copy code to clipboard mod
static $codecounter = 0;
$codecounter++;
$tag[\'content\'] = "<div class=\"codeheader\">" . $txt[\'smf238\'] . ":" . $data[1] . " <a href=\"javascript:void(0)\" title=\"" . $txt[\'codeselectall\'] . "\" onclick=\"javascript:selectall(\'code_a" . $codecounter . "\');return false;\">" . $txt[\'codeselectall\'] . "</a>
</div>' . $codeWrapHTML . ';
'),
'block_level' => true,
);
$codes[] = array(
'tag' => 'code',
'type' => 'unparsed_equals_content',
'validate' => isset($disabled['code']) ? null : create_function('&$tag, &$data, $disabled', '
global $txt;
// select all and copy code to clipboard mod
static $codecounter = 0;
$codecounter++;
$tag[\'content\'] = "<div class=\"codeheader\">" . $txt[\'smf238\'] . ":" . $data[1] . " <a href=\"javascript:void(0)\" title=\"" . $txt[\'codeselectall\'] . "\" onclick=\"javascript:selectall(\'code_b" . $codecounter . "\');return false;\">" . $txt[\'codeselectall\'] . "</a>
</div>' . $codeWrapHTML . ';
'),
);


[Blizzard]

I'll have to look into that another time. Judging by the code, I'm not even sure that feature is actually supported.

EDIT: I'll add that code right away. Let's see if it works.

EDIT: And you broke my selection code, Ryex.

EDIT: Alright, I added your code (with some modifications), but I don't think it's working.

[Ryex]

it doesn't always pick the right syntax sadly but most of the time it does and when it doens't half the time the addition of the right code type will help it pick. actually you might want to put an extra space after the :

[Blizzard]

Sure, if it helps.

EDIT:

Code: (ruby) [Select]

int main()
{
    printf("Hellooooooooooooooooooooo.\n");
    return 0;
}

Code: (cpp) [Select]

int main()
{
    printf("Hellooooooooooooooooooooo.\n");
    return 0;
}
EDIT: I think all it really does is detect keywords, comments and stuff enclosed in quotes, etc.

EDIT: I updated Subs-Highlight.php in my DB in case you wanna use it. But be careful. I removed some stuff that I don't need.

[Ryex]

I figured it out. if you explicitly set the class of the code tag it uses that highlighting
also only the

Code: [Select]

'type' => 'unparsed_equals_content', should set the code header I realized

Code: (php) [Select]

// Code tag
function ch_code_tag(&$codes)
{
    global $modSettings, $txt, $settings;

    $darkThemeIDS = array(3, 5, 7, 9, 11, 13);

    if (in_array($settings['theme_id'], $darkThemeIDS))
    {
        $codeWrapHTML1 = '<div class=\"block_code\" id=\"code_a" . $codecounter . "\"><div class=\"dark_theme_code\"><pre><code>$1</code></pre></div></div>"';
        $codeWrapHTML2 = '<div class=\"block_code\" id=\"code_b" . $codecounter . "\"><div class=\"dark_theme_code\"><pre><code class=\"" . $data[1] . "\">$1</code></pre></div></div>"';
    }
    else
    {
        $codeWrapHTML1 = '<div class=\"block_code\" id=\"code_a" . $codecounter . "\"><pre><code>$1</code></pre></div>"';
        $codeWrapHTML2 = '<div class=\"block_code\" id=\"code_b" . $codecounter . "\"><pre><code class=\"" . $data[1] . "\">$1</code></pre></div>"';
    }

    if (!empty($modSettings['ch_enable']))
    {
        foreach ($codes as $tag => $dump)
            if ($dump['tag'] == 'code') unset($codes[$tag]);
            
        $codes[] =  array(
            'tag' => 'code',
            'type' => 'unparsed_content',
            'validate' => isset($disabled['code']) ? null : create_function('&$tag, &$data, $disabled', '
                global $txt;
                // select all and copy code to clipboard mod
                static $codecounter = 0;
                $codecounter++;
                $tag[\'content\'] = "<div class=\"codeheader\">" . $txt[\'smf238\'] . ": <a href=\"javascript:void(0)\" title=\"" . $txt[\'codeselectall\'] . "\" onclick=\"javascript:selectall(\'code_a" . $codecounter . "\');return false;\">" . $txt[\'codeselectall\'] . "</a></div>' . $codeWrapHTML1 . ';
            '),
            'block_level' => true,
        );
        $codes[] = array(
            'tag' => 'code',
            'type' => 'unparsed_equals_content',
            'validate' => isset($disabled['code']) ? null : create_function('&$tag, &$data, $disabled', '
                global $txt;
                // select all and copy code to clipboard mod
                static $codecounter = 0;
                $codecounter++;
                $tag[\'content\'] = "<div class=\"codeheader\">" . $txt[\'smf238\'] . ": " . $data[1] . " <a href=\"javascript:void(0)\" title=\"" . $txt[\'codeselectall\'] . "\" onclick=\"javascript:selectall(\'code_b" . $codecounter . "\');return false;\">" . $txt[\'codeselectall\'] . "</a></div>' . $codeWrapHTML2 . ';
            '),
            'block_level' => true,
        );          
    }
    
    // Copyright Info
    if (isset($context['current_action']) && $context['current_action'] == 'credits')
        $context['copyrights']['mods'][] = '<a href="http://dragomano.ru/page/code-highlighting" target="_blank">Code Highlighting</a> &copy; 2010&ndash;2012, Bugo';
}


tags you can use for code highlighting:

• python
• profile (like output form python's profiler)
• ruby
• perl
• php
• scala
• go
• xml (also supports HTML with inline css and javascript)
• markdown
• django (Django templates)
• css
• json
• javascript
• coffeescript
• actionscript
• vbscript
• http (like output from curl the standard way HTTP headers as displayed in text)
• lua
• applescript
• delphi
• java
• cpp (c++)
• objectivec
• vala
• cs (c#)
• d
• rsl (RenderMan RSL)
• rib (RenderMan RIB)
• mel (Maya Embedded Language)
• glsl
• sql
• smalltalk
• lisp
• clojure
• ini
• apache (like .htaccess)
• nginx
• diff (SVN or GIT diffs ect.)
• dos (DOS batch files)
• bash (.sh scripts)
• cmake
• axapta
• 1c
• avrasm (AVR Assembler)
• vhdl
• parser3
• tex (TeX)
• brainfuck (BrainFuck)
  
• haskell
• erlang
• erlang-repl
• rust
• matlab
• r
• no-highlight (disables highlighting)

You might want to update the actual highlight script too as I've noticed there has been a few updates since this this mode went out.
https://dl.dropbox.com/u/20709011/highlight.pack.js

[Blizzard]

Code: [Select]

class A; end;

Code: (test) [Select]

class A; end;

Code: (Ruby) [Select]

class A; end;

Code: (ruby) [Select]

class A; end;
EDIT: It seems to be working except for one bug. When I used code tags without the =something part, then a weird L appears there (first example). You think you can fix that, Ryex?

[Ryex]

I already did in the code I posted. as I said the array with the type "unparsed_content" SHOULD NOT append the $data[1] part after the "Code:" text. only the "unparsed_equals_content" should

also, did you update the Highlight script like I suggested at the end of my last post?

I appears that as long as you use one of the tags I posted it will use that but if it doesn't recognize the tag it will auto detect anyway.

[Blizzard]

Ah, I see now *fixes* and yes, I did.

[Ryex]

well folks. there you go, one fancy Syntax Highlighting system fresh off the presses.

you may bow and bequeath us with favors as time permits.
shouts of adoration are appreciated but we, for the most part, can not return your feelings.

[Blizzard]

can not return your feelings.

Because we are robots.

[Ryex]

I like how when I go into the script help section and I see a code tag it's all highlighted and stuff, makes my want to do a fist pump because my brain is like "OMG YES THIS IS SO EASY TO READ!"

[Blizzard]

I just noticed that it seems to often fail to recognize the language, especially in short code segments. I think I'll set it to Ruby by default and then using code=something would allow different highlighting for other languages.

EDIT: Wow, this was frustratingly difficult. Not because it was a difficult thing to do, but because the javascript code looks like it's been horribly unreadable on purpose. :/ The variable names are usually just one letter and the entire thing is in one single line with no spaces. Anyway, this is how it's done.

There should be a code segment like this somewhere in the beginning:

Code: (javascript) [Select]

if(e[p[o]]||p[o]=="no-highlight"){return p[o]}
When this is added at the end, it causes the certain language to become default if none was specified:

Code: (javascript) [Select]

if(p[o]==""){return "ruby"}
EDIT: Hey, Ryex, there seems to be an error in the Ruby parser. Check out this script:

http://forum.chaos-project.com/index.php/topic,2312.msg174554.html#msg174554

Notice how the parameters labeled x0, y0, etc. have their numbers highlighted as if they were literals. This is wrong. :/ You think you can fix that?

[Ryex]

ya no. the highlight.js parser isn't easy to follow the code is a bit wonky. you can try seeing if there is a new update for it though. (checks himself.) nope last update was 13.10.2012

[Blizzard]

See if this can at least be reported.

[Ryex]

There has been an update to Highlight.js http://highlightjs.org/
Release 8.0 https://github.com/isagalaev/highlight.js/releases/tag/8.0

• auto detection has been improved
• a bunch of styles have been improved (including ruby's)
• new languages added
• language aliases have been added (ie `js` for javascript and `html` which previously had to xml)

and a bunch more.

update the .js file if you would please Blizz.

[Blizzard]

I remember that I modded some things in the script. I don't think it will be just copy-paste, but I'll check it out.

EDIT: Nope, I simply can't get it to work. Replacing doesn't help and there are no instructions as to where to put the extra folders or if they need to be loaded separately. I tried using the prebuilt version, it didn't work either.

[Ryex]

use this https://dl.dropboxusercontent.com/u/20709011/Random%20Stuff/highlight.zip

you link the highlight.pack.js, link one of the styles from the style directory and you should be good to go.

[Blizzard]

That might work. The old file I have is also called highlight.pack.js. I'll try it out when I get home from work.

EDIT: Nope, I'm still having trouble to get things running. It removes the select-code button and just turns everything into black text. I can give you CP's source if you feel like messing around with it.

[Ryex]

alright. time to mess around with this. toss it here Blizz. point me to the files in question (i've kinda forgotten )

[Blizzard]

Alright, I'll send it to you when I get home.

[Ryex]

done, check PM Blizz.

try useing this to help you select styles for light and dark themes instead of just edit test repeating
https://highlightjs.org/static/test.html

notice that you can select the them at the top

also this lists all the languages the highlighting supports as well as what aliases you can put in the

Code: [Select]

code=<lang> tag to force highlighting in a language. if there is enough of it the language should be auto detected but shot one liners might need some help.


Tomorrow Night Bright looks like a good dark theme

[Blizzard]

Added.

[Ryex]

Something is wrong, this is what it was dining to me before i stopped it from useing the old css. Something is preventing the css file from loading

[Blizzard]

Try refreshing the site, clearing your cache and/or restarting your browser. When I first uploaded everything, it was totally broken. Then I pressed F5 and it was fine. Also, I removed the highlight.css in the default folder since it wasn't even being used anymore.

[Ryex]

oh ok ya it's working now, I did some edits, it's easier to configure now as you only have to change the variables as the top of the script.
also
CODE IS PROPERLY FORMATED DURING A PREVIEW!

http://pastebin.com/wsbvtTFC

EDIT: I tried putting it in a code tag but the the existence of a code tag in the source just made it drop some of the script

[Blizzard]

Finally, lol! I was already looking everywhere where vs.css was defined. xD I uploaded the new one.

[Ryex]

ah crap I forgot the most important part


replace

Code: (php) [Select]

   // Preview
    if (!empty($modSettings['ch_enable']) && in_array($context['current_action'], array('post', 'post2')))
        $context['html_headers'] .= '
        <script type="text/javascript">
            var highlightPreviewCodeBlocks = function(){
                if (highlightPreviewCodeBlocks.calling)
                    return;
                highlightPreviewCodeBlocks.calling = true;
                var blocks = document.querySelectorAll("#preview_body pre code");
                Array.prototype.forEach.call(blocks, hljs.highlightBlock);
                highlightPreviewCodeBlocks.calling = false;
            };
            
            document.querySelector("#preview_body").addEventListener("DOMSubtreeModified", highlightPreviewCodeBlocks, false);
        </script>';

with

Code: (php) [Select]

   // Preview
    if (!empty($modSettings['ch_enable']) && in_array($context['current_action'], array('post', 'post2')))
        $context['html_headers'] .= '
        <script type="text/javascript">
            var highlightPreviewCodeBlocks = function(){
                if (highlightPreviewCodeBlocks.calling)
                    return;
                highlightPreviewCodeBlocks.calling = true;
                var blocks = document.querySelectorAll("#preview_body pre code");
                Array.prototype.forEach.call(blocks, hljs.highlightBlock);
                highlightPreviewCodeBlocks.calling = false;
            };

            var initPreviewHighlighting = function(){
                if (initPreviewHighlighting.called)
                    return;
                initPreviewHighlighting.called = true;
                document.querySelector("#preview_body").addEventListener("DOMSubtreeModified", highlightPreviewCodeBlocks, false);
            };

            var initPreviewHighlightingOnLoad = function(){
                document.addEventListener("DOMContentLoaded", initPreviewHighlighting, false);
                document.addEventListener("load", initPreviewHighlighting, false);
            };
           
            initPreviewHighlightingOnLoad();
           
        </script>';

otherwise the code to highlight code in the preview section when you click the preview button doesn't get attached as the preview section doesn't exist yet.

[G_G]

Absolutely loving it. Awesome job Ryex.

[WhiteRose]

Is it supposed to look like this, or is something not working properly for me?

(click to show/hide)

[KK20]

It looked like that for me the first time too. But refreshing did the trick. It's one of those many mysteries of the internet~

[Ryex]

if it looks like that It means it' cached the old highlight.pack.js/css files and you need to force the page to refresh.

[Blizzard]

The same happens for me briefly after loading/reloading the site, just before the highlighting is applied. Even with the new code.

[Blizzard]

*double post* Ryex, something's wrong. Check out the script. Its line breaks are broken.

http://forum.chaos-project.com/index.php/topic,4284.0

[Ryex]

it's auto detecting as elixir instead of ruby. we should probably go and generate a Highlight.pack.js that doesn't include every language possible to avoid it detecting the weird languages.

we should probably go here and build a version with less than all the languages
https://highlightjs.org/download/

[Blizzard]

Done.

[Ryex]

I figured out why we get the weird formatting on non highlighted blocks of text (remember you can turn off hilightinging by useing code=nohighlight or code=no-highlight)
you need to add

Code: (nohighlight) [Select]

display: block;

to all your

Code: (css) [Select]

.code, pre.code {
...
}

blocks in your theme css files

[Blizzard]

Done.

[G_G]

Dunno if it's something that can be fixed, but it doesn't seem to be liking the tab character.

You can clearly see there's a tab.


But it won't show it in the post.


Making scripts look like garbage.

[Blizzard]

IDK. ._.;

[Ryex]

it's because the web in general doesn't like the tab character there is no standardizes support for it's display.

replacing with spaces might be the only option

[G_G]

Would it be possible to use regexp and sub out the tab "\t" and replace with like 4 spaces "&nbsp;"?

Meant to originally post this here.

[Ryex]

LOL!
turns out this functionality is already implemented.

there is a mod setting to configure how many space tabs a space gets replaced with
and this code is supposed to add that many spaces together to set up the tab replace.

this sequence should appear twice in Subs-Highlight.php

not sure why it's not working.

Code: [Select]

   $tab = '';
    if (!empty($modSettings['ch_tab'])) {
        while ($i < $modSettings['ch_tab']) {
            $tab .= ' ';
            $i++;
        }
    }
    // ...
    $js = "\n\t\t" . '<script type="text/javascript" src="' . $settings['default_theme_url'] . '/highlight.pack.js"></script>
        <script type="text/javascript">
        hljs.configure({
          tabReplace: "' . $tab . '",
          useBR: true
        });
        hljs.initHighlightingOnLoad();
        </script>';


[Blizzard]

Nope, not there AFAIK.

(click to show/hide)

[Ryex]

Number of characters between tab columns [ 4 ]

^^

[Blizzard]

Wouldn't that mean it's already converted to spaces?

[Ryex]

it's supposed to yes, I'm not sure why it's not working.

[Blizzard]

Any chance you can find some time to look into it?