View Issue Details

IDProjectCategoryView StatusLast Update
0005331unrealircdpublic2019-08-18 15:17
ReporterGottemAssigned ToGottem 
Status resolvedResolutionfixed 
Product Version 
Target Version5.0.0-alpha1Fixed in Version5.0.0-alpha1 
Summary0005331: Port m_storetkl
<Syzop> I see you write them in binary and then have to resolve them when reading back to characters or strings.. so I would do that the other way around.. writing them that way :)
<Syzop> and then (different situation): we have some timestamps, tkl->expire_at, tkl->set_at, tkl->ptr.spamf->tkl_duration, we could write some general 64 bit writing/reading int function for that I think

1.1) subtype is currently written and read but not passed along to anywhere. :D It might be necessary for softbans or it might simply be duplicate information. spamfilter_target_inttostring(tk->subtype) resolves to a target string but I'm using tk->usermask which holds the exact same information most likely.

1.2) Speaking of softbans, make sure that shit works/keeps working after re-adding em.

1.3) Instead of writing on every TKL_ADD/DEL, use an EVENT to do it every X minutes (configurable?) and prolly on MOD_UNLOAD too.

1.4) Write to a tempfile first and if all good then move it to the "real" path.

1.5) Skip tklines with flag 'f' (built-in spamfilters or added through a .conf) on write too.

1.6) Fix memory leak when readDB() returns -1 mid-way.

1.7) For Z:Lines it should prolly use the new hash thingy instead of indices. =]

1.8) Rename m_storetkl to tkldb.
TagsNo tags attached.
3rd party modules


child of 0005282 assignedGottem Gottem's todo list yo 



2019-07-13 19:07

developer   ~0020781

1.0) It now *writes* the result of tkl_typetochar(), so I can simply read it back into a char *tkltype and get just the flag with tkltype[0]. I could write the singular char (as in 'G' instead of "G\0") but I need to pass it as a char * in parv[] anyways. :D I also changed some of the size_t types to uint64_t since evidently it's not a very good idea to write the former to binary files. The uint64_t is used for storing strlen() on write_str() and for reading it back, as well as the count of TKLines. Since the module itself controls all of these, this seems the best way. =] On my system both size_t and uint64 are 8 bytes long anyways, but for 32-bit compilers size_t might be 4 in which case it corresponds to just an uint32.

As for the timestamp stuff you mentioned, how were you imagining that? I've thought about storing it in a uint64_t before writing, then read it back into that same type before converting it along to TS (which eventually is just an int32_t I think?). Since TS is less significant you'd lose a couple bytes in the process, but you wrote a TS to begin with so it wouldn't/shouldn't matter. Then if you decide to upgrade TS to int64_t there's no bit loss anymore. Signedness doesn't matter afaict because you still have the same amount of bytes, just treated a little differently. I also don't think it's possible to have negative timestamps or TKL durations. :>

The backwards compatibility code is also a little ugly, but it could've been much worse. :D Just search for "tkl1000" and you'll easily find all the relevant stuff.

1.1) spamfilter_target_inttostring(tk->subtype) and tk->usermask are indeed exactly the same, meaning subtype is only used for softbans at the moment. So for spamfilters I'll simply keep using the already-read usermask and only check subtype for softbans.

1.2) Softbans actually DIDN'T work, as the % was lost entirely. :D So when writing TKLs, I now do:
char usermask_subtype[256];
if(!tkl->ptr.spamf && (tkl->subtype & TKL_SUBTYPE_SOFT)) {
    snprintf(usermask_subtype, sizeof(usermask_subtype), "%%%s", tkl->usermask);
    W_SAFE(write_str(fd, usermask_subtype));
    W_SAFE(write_str(fd, tkl->usermask));

I figured it's better to not snprintf() if it's not a soft ban to begin with, I guess this saves some precious microseconds. :D

1.3) Added the event with an interval of 893 seconds.

1.4) Dunzo

1.5) Little bonus I noticed while working on above:
I was already skipping flag 'f' (built-in spamfilters or added through a .conf) on read, but it now also does so on write.

1.6) A bigger bonus:
The R_SAFE macro makes readDB() return -1 on read errors, but it didn't free the strings that might have been malloc'd in the meantime (tkltype, usermask, hostmask, reason, setby, spamf_check). :>


Woop dee do that's a bloody wall of text. Guess you won't be needing your morning newspaper tomorrow. :D


2019-07-13 19:07

developer   ~0020782

Kkkkk so storetkl now stores TS in uint64_t instead. =] *@IP Z:Lines are also checked according to the hash stuff that's going on.

Also renamed m_storetkl to just tkldb cause I think we agreed on that. I just can't find the goddamn logs. ;_;


2019-07-13 19:07

administrator   ~0020783

I did some re-indenting and some cleanups, see last few commits.
I should probably also have pointed you to doc/coding-guidelines which documents most of these. Seems a bit outdated, best to start reading at point 10 in that document, the ones before that need to be updated :D

Issue History

Date Modified Username Field Change
2019-07-13 19:07 Gottem New Issue
2019-07-13 19:07 Gottem Status new => assigned
2019-07-13 19:07 Gottem Assigned To => Gottem
2019-07-13 19:07 Gottem Issue generated from: 0005282
2019-07-13 19:07 Gottem Note Added: 0020781
2019-07-13 19:07 Gottem Note Added: 0020782
2019-07-13 19:07 Gottem Note Added: 0020783
2019-07-13 19:07 Gottem Relationship added child of 0005282
2019-07-13 19:10 Gottem Status assigned => resolved
2019-07-13 19:10 Gottem Resolution open => fixed
2019-07-13 19:10 Gottem Fixed in Version => 5.0.0-alpha1
2019-08-18 15:17 syzop View Status private => public