stet/keith question or whoever is in charge of code these days

Can you remember exactly what you were searching for? Email me if you like - stet @ ilxor

Odd - thread searches should be markedly quicker than post searches!

If it's a problem, we might need to just rebuild the search index from scratch.

i get slow searches sometimes too

i mentioned it to stet the first time

it happens every so often but i'll post when it does

Can you remember exactly what you were searching for? Email me if you like - stet @ ilxor

― stet, Wednesday, September 14, 2016 1:21 PM (twenty-two minutes ago) Bookmark Flag Post Permalink

i was searching for the 2015 end of year tracks poll. I think I searched "ilm 2015", "ilm 2015 tracks", "ilm tracks" etc and selected "most recent first" -- each time the search would come up but as I clicked through and approached the matching threads from january clicking "next" wouldn't work. for some of the searches this was on the third page, for another it was the fourth, another the second...it was bizarre, almost like the search knew what thread I wanted to find and didn't want me to see it

I keep searching for "; drop table users;" and it doesn't work!!

ha

sorry, I really tried not to type that

it was quicker than trying to roll together a joke about apache commons problems with deserialization

Haha (injection thing).

(Yes, they're not going to work)

It's possible that what's happening is a function of how it works and housekeeping...

The way it works is that I run a big job at the beginning that scans all the threads and posts and creates two index files for the search: one for threads, one for posts. The threads one is miles smaller; hence, why it should be faster. With hindsight, I wouldn't even have bothered with the threads one, as subsequent to developing the search, I decided it was useful to just hold the whole list of threads in memory at all times anyway for fast access.

Anyway, that's cool for startup, but the way it keeps the index up-to-date with changes since the initial build is that a batch job runs every five minutes, scans the database for new or changes to threads and posts and updates the index.

I'd have thought that ultimately, this would fragment the index file and make it slow and I for one certainly haven't recreated the index file for at least five years, which would be half a million updates to the index files at least! So guessing somewhat here, but if Stet hasn't recreated the index file since I last did, which I might doubt, as I don't think we ever talked about what that thing did before, then it's possible that's what's going on and all we need to do is stop the site, recreate the index file and start it again.

It's actually an all-new index file as of a couple of weeks ago, so I'm wondering if something went iffy there. No harm in making a new one, will kick that off later.

> 19:17 (fifty-six minutes ago)

It is 21:11 here now. Why is that saying 56 minutes?

>

> 20:14 (three minutes ago)

Slightly better

Although it's 21:18 here, not 20:18

Someone else doing a big search

The index file could potentially be screwed if it were created when the site was running when that happened. I mean, I don't know whether it would or not, but I've never thought through whether it would cause issues. Certainly the initial job reads big fuckoff lumps of the database at a time and indexes that then moves on to the next; the database changing under its feet could render some of the already-created index incorrect, although it'd be up to Apache Lucene whether or not that cause the whole thing problems or not.

Am I right in saying this was all TEN YEARS AGO now?

is it just me or is the spacing all screwed up on the search page, like the css isn't loading right or something

Am I right in saying this was all TEN YEARS AGO now?

― Keith, Wednesday, 14 September 2016 22:32 Bookmark

no the right way to say it is 10 years ago now

Well clearly ten years on this place hasn't changed.

I just bought a new Android tab, and now I'm seeing banner ads on ILX even I log in with Chrome. The tablet has Android 6.0, I didn't seen any ads while using Chrome on my older tablet, which had an older version of Android.

"even I after log in"

http://i1326.photobucket.com/albums/u641/Lixenixen/Mobile%20Uploads/Screenshot_20161007-192926_zpsce6zozqq.png

Here's an example, I'm logged in, but I still see the as.

not sure why, but half of the things I search for cause ILX to hang.

for instance, I searched for Lost Boys, and it comes up no problem.

I search for any variant of "let's shit our pants to something new", even without the word 'shit', and none of them work, except "Let's shit", and doesn't return that thread at all. I tried "our pants", "something new", "shit our pants", and it just hung for minutes at a time.

Don't know if it's related, but threads have been taking a long time to load the last couple of days.

help! the site is doing that thing where every last bookmark I've ever had is showing up at the top of SNA and I can't make them go away. I tried changing the Use Traditional Bookmarks setting, but it didn't do anything. How do I fix this? What am I doing to cause this?

hmmm, well I think I fixed this for now. I changed the Google account in my Chrome browser and that seemed to help, not sure why.

the search function is still super buggy and slow fyi

my versh of zing suddenly won't load any URLs, unless I jump out to safari. i.e. won't display em on zing. is it just me?

Even though I'm logged in, I still see a banner ad on this thread:

Metallica ARE...Hardwired…To Self-Destruct

It doesn't matter much to me, but it's strange that's the only thread (so far) where that happens.

FYI

fuckin' lars

I get banner ads on every thread while logged in on my work computer.

the search function is still super buggy and slow fyi

― k3vin k., Sunday, November 13, 2016 2:01 PM (four days ago)

is this just me? it's been going on for months. i can't find threads i want to find

no prob for me

I get banner ads on every thread while logged in on my work computer.

This keeps happening to me a lot when I use Chrome on Android, as I mentioned above. Not on my home tabletop computer though (I use Firefox with Adblock Plus).

search: it depends on what you search.

e.g. search for Zimmerman: it hangs forever/timeout
search for Trayvon: it hangs forever/timeout

search for energy: immediate results
search for zooropa: immediate results (it's not that the end of the alphabet is slower)
search for Ziegler: immediate result (it's not last names starting with capital letters either)

^ googleproofed = no search results? But then I DID find that one thread I couldn't find here by going to google and searching for the same thing and site:ilxor.com so I don't know

sounds like it might be thread specific causing problems

yeah my search problems vary depending on the search term

Have rebuilt the indexes the search system uses. Zimmerman and Trayvon now both working for me; can people try things that were broken for them?

I still can't get "clickhole" to search

"trayvon" still broken for me too

searching "2015 albums" also not working -- this is what brought this to my attention a few weeks ago when i was trying to find the year end poll

Let me check i'm doing the same search as you — when I try it's instant. I'm searching:

http://i.imgur.com/f4Zrvfk.png

i'm doing thread titles, not posts. i also was not using quotes -- however when i do use quotes (and search most relevant, all boards) it works!

just tried again. 2015 albums, searching thread titles. quotes = works. no quotes = doesn't work

i use google site search

The search engine is Apache Lucene. Whatever works with Lucene will work with it. I didn't do anything special on top of it. Here is a page that describes some of the stuff you ought to be able to do, although to be fair, it may be that it does more now than it did back when this stuff was done:

https://lucene.apache.org/core/2_9_4/queryparsersyntax.html