CRUCIVERB.COM

User

Welcome, Guest.
Please login or register.
 
 
 
Forgot your password?

Navigate

Resources

Donations


You can help support this site by making a small donation using either a PayPal account:

or with a major credit card such as:

 

 

Click here for details.

Author Topic: How many words should be in a word list?  (Read 34739 times)

zvit

  • Newbie
  • *
  • Posts: 25
How many words should be in a word list?
« on: October 31, 2021, 07:07:29 AM »
I tried to add two dictionaries in addition to the "default" one in Crossfire, and it wouldn't accept the third one, saying it was out of memory. It's not my computer memory because I have 128GB of ram and most of it is free when I get this error. I'm guessing it's the software's database limitation but don't understand why.

The Crossfire default dictionary, together with my two dictionaries combined, has 979,211 words.

This got me thinking: I see that the more words you let a software work with, the more autofill options it finds. Too little a word list and the software will say "unfillable grid."

But if 979,211 words spit out a memory error, I would like to know how many words do you guys typically let a software work with?


mmcbs

  • Hero Member
  • *****
  • Posts: 520
Re: How many words should be in a word list?
« Reply #1 on: October 31, 2021, 07:29:35 AM »
I'm not a Crossfire used, but 900K+ words is larger than any viable word list I've heard of. I suspect you have some duplicates in there, perhaps phrases spaced differently such as HELP OUT and HELPOUT, or some that are the same but one in lower case the other in upper case, or some punctuated and others not. Crossword Compiler has a utility to merge these, but I don't know about Crossfire.
Mark McClain
Salem, Virginia, USA
https://crosswordsbymark.wordpress.com/

zvit

  • Newbie
  • *
  • Posts: 25
Re: How many words should be in a word list?
« Reply #2 on: October 31, 2021, 07:53:19 AM »
I'm not a Crossfire used, but 900K+ words is larger than any viable word list I've heard of. I suspect you have some duplicates in there, perhaps phrases spaced differently such as HELP OUT and HELPOUT, or some that are the same but one in lower case the other in upper case, or some punctuated and others not. Crossword Compiler has a utility to merge these, but I don't know about Crossfire.

Thanks, but nope, I don't have duplicates as you mentioned. I use excel to delete duplicates and check that my list only has one-word words. (screenshot below)

Being new to compiling, and noticing that the more words I had, the more autofill options I got and thinking that professionals probably use millions of words, I just added more and more.

So my question is, what IS a normal word list number? =<10K? =<100K?

And WHY wouldn't you use a million words if you get more autofill results? I know professionals don't autofill ever puzzle but the autofill gives you a good base to start with and good ideas.





mmcbs

  • Hero Member
  • *****
  • Posts: 520
Re: How many words should be in a word list?
« Reply #3 on: October 31, 2021, 08:08:05 AM »
Well, of the screenshot you provided, about half are words that would never be used in a mainstream crossword, and unfortunately this is typical of the "big" word lists. They're just full of technical or specialized words/phrases that are not known by most people. They are chock full of abbreviations that are not familiar. They have many offensive, inappropriate, or outdated words/phrases. When it comes to word lists bigger is not better. My "default" word list is about 400,000 and probably less than half of it is usable in a typical crossword. To create a grid that will be enjoyable for your solvers (or acceptable to an editor) you have to be very selective about what words you use, and this is why professionals never use autofill unless they have a 100% curated word list. I actually have one of these that I call Basic English which has under 40,000 entries.
Mark McClain
Salem, Virginia, USA
https://crosswordsbymark.wordpress.com/

zvit

  • Newbie
  • *
  • Posts: 25
Re: How many words should be in a word list?
« Reply #4 on: October 31, 2021, 08:34:21 AM »
Thanks. By the way, I am not creating a mainstream crossword, it's for the medical community (ems, doctors, nurses, paramedics, etc.) that's why my list is unique.

asternknit

  • Newbie
  • *
  • Posts: 7
Re: How many words should be in a word list?
« Reply #5 on: October 12, 2023, 12:38:41 AM »
Thanks. By the way, I am not creating a mainstream crossword, it's for the medical community (ems, doctors, nurses, paramedics, etc.) that's why my list is unique. geometry dash world
They simply contain a lot of unfamiliar technical or specialist terminology and phrases. They are jam-packed with unfamiliar abbreviations. They use a lot of language that is offensive, improper, or out of date. Bigger is not always better when it comes to word lists.
 

evawillms

  • Newbie
  • *
  • Posts: 2
Re: How many words should be in a word list?
« Reply #6 on: April 02, 2024, 11:46:28 PM »
I tried to add two dictionaries in addition to the "default" one in Crossfire, and it wouldn't accept the third one, saying it was out of memory. It's not my computer memory because I have 128GB of ram and most of it is free when I get this error. I'm guessing it's the software's database limitation but don't understand why.

The Crossfire default dictionary, together with my two dictionaries combined, has 979,211 words.

This got me thinking: I see that the more words you let a software work with, the more autofill options it finds. Too little a word list and the software will say "unfillable grid."

But if 979,211 words spit out a memory error, I would like to know how many words do you guys typically let a software work with? Run 3
In the case of word games or puzzle-solving software like Crossfire, the software needs to maintain a word list in memory to generate autofill options and verify word validity. The larger the word list, the more options the software can generate. However, a larger word list also requires more memory to store and process.

jopela

  • Newbie
  • *
  • Posts: 7
Re: How many words should be in a word list?
« Reply #7 on: June 27, 2024, 03:41:24 AM »
Quote
In the case of word games or puzzle-solving software like Crossfire, the software needs to maintain a word list in memory to generate autofill options and verify word validity. The larger the word list, the more options the software can generate. However, a larger word list also requires more memory to store and process.snow rider 3d
I completely agree with this idea, the word list must be balanced so that the memory can store enough.

aryastark

  • Newbie
  • *
  • Posts: 1
Re: How many words should be in a word list?
« Reply #8 on: August 01, 2024, 03:48:47 AM »
The optimal size of a word list depends on the software's capacity. In your case, Crossfire encountered a memory error at around 979,211 words, likely due to its database limits. A larger list provides more autofill options, but too many words can cause issues. Try different sizes to find the best balance for your needs.
my perfect hotel
« Last Edit: August 06, 2024, 08:56:39 PM by aryastark »

allanpetter

  • Newbie
  • *
  • Posts: 12
Re: How many words should be in a word list?
« Reply #9 on: August 02, 2024, 11:46:31 PM »
It sounds like you've run into a limitation of the Crossfire software itself rather than an issue with your computer's memory. Many software applications have their internal limits on how much data they can handle, which might be what you're experiencing.
Lone Star Home Remodeling Pros Fort Worth tx

metalcraft whangarei
« Last Edit: September 10, 2024, 06:41:51 AM by allanpetter »

 


Powered by EzPortal